2023-02-13 00:53:35,665 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-02-13 00:53:35,940 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79 2023-02-13 00:53:35,954 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-13 00:53:35,954 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/cluster_0ce39707-0279-6124-1e28-d0d9ced157d1, deleteOnExit=true 2023-02-13 00:53:35,955 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-13 00:53:35,955 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/test.cache.data in system properties and HBase conf 2023-02-13 00:53:35,956 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/hadoop.tmp.dir in system properties and HBase conf 2023-02-13 00:53:35,957 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/hadoop.log.dir in system properties and HBase conf 2023-02-13 00:53:35,957 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-13 00:53:35,958 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-13 00:53:35,958 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-13 00:53:36,077 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-13 00:53:36,477 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-13 00:53:36,482 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-13 00:53:36,483 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-13 00:53:36,483 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-13 00:53:36,484 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 00:53:36,484 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-13 00:53:36,484 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-13 00:53:36,485 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 00:53:36,485 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 00:53:36,486 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-13 00:53:36,486 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/nfs.dump.dir in system properties and HBase conf 2023-02-13 00:53:36,486 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/java.io.tmpdir in system properties and HBase conf 2023-02-13 00:53:36,487 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 00:53:36,487 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-13 00:53:36,487 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-13 00:53:37,080 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 00:53:37,085 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 00:53:37,397 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-13 00:53:37,586 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-13 00:53:37,606 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 00:53:37,649 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 00:53:37,682 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/java.io.tmpdir/Jetty_localhost_46367_hdfs____nl6524/webapp 2023-02-13 00:53:37,844 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46367 2023-02-13 00:53:37,883 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 00:53:37,883 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 00:53:38,342 WARN [Listener at localhost/38857] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 00:53:38,472 WARN [Listener at localhost/38857] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 00:53:38,495 WARN [Listener at localhost/38857] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 00:53:38,503 INFO [Listener at localhost/38857] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 00:53:38,509 INFO [Listener at localhost/38857] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/java.io.tmpdir/Jetty_localhost_44171_datanode____by7836/webapp 2023-02-13 00:53:38,620 INFO [Listener at localhost/38857] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44171 2023-02-13 00:53:38,995 WARN [Listener at localhost/37869] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 00:53:39,554 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x62afee76db9050ab: Processing first storage report for DS-bcb7cc95-c419-4075-9477-71577858f0ce from datanode 2310d9d9-ed70-437e-8abe-6c0931ad74c5 2023-02-13 00:53:39,556 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x62afee76db9050ab: from storage DS-bcb7cc95-c419-4075-9477-71577858f0ce node DatanodeRegistration(127.0.0.1:36491, datanodeUuid=2310d9d9-ed70-437e-8abe-6c0931ad74c5, infoPort=39789, infoSecurePort=0, ipcPort=37869, storageInfo=lv=-57;cid=testClusterID;nsid=604844353;c=1676249617158), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-02-13 00:53:39,556 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x62afee76db9050ab: Processing first storage report for DS-1194e2c7-7c8f-4435-ade7-8f7be2fc2be7 from datanode 2310d9d9-ed70-437e-8abe-6c0931ad74c5 2023-02-13 00:53:39,556 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x62afee76db9050ab: from storage DS-1194e2c7-7c8f-4435-ade7-8f7be2fc2be7 node DatanodeRegistration(127.0.0.1:36491, datanodeUuid=2310d9d9-ed70-437e-8abe-6c0931ad74c5, infoPort=39789, infoSecurePort=0, ipcPort=37869, storageInfo=lv=-57;cid=testClusterID;nsid=604844353;c=1676249617158), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 00:53:39,592 DEBUG [Listener at localhost/37869] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79 2023-02-13 00:53:39,986 INFO [Listener at localhost/37869] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/cluster_0ce39707-0279-6124-1e28-d0d9ced157d1/zookeeper_0, clientPort=54478, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/cluster_0ce39707-0279-6124-1e28-d0d9ced157d1/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/cluster_0ce39707-0279-6124-1e28-d0d9ced157d1/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-13 00:53:39,999 INFO [Listener at localhost/37869] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=54478 2023-02-13 00:53:40,007 INFO [Listener at localhost/37869] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 00:53:40,008 INFO [Listener at localhost/37869] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 00:53:41,010 INFO [Listener at localhost/37869] util.FSUtils(479): Created version file at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00 with version=8 2023-02-13 00:53:41,010 INFO [Listener at localhost/37869] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/hbase-staging 2023-02-13 00:53:41,327 INFO [Listener at localhost/37869] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-13 00:53:41,773 INFO [Listener at localhost/37869] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=3 2023-02-13 00:53:41,804 INFO [Listener at localhost/37869] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 00:53:41,805 INFO [Listener at localhost/37869] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 00:53:41,805 INFO [Listener at localhost/37869] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 00:53:41,805 INFO [Listener at localhost/37869] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 00:53:41,806 INFO [Listener at localhost/37869] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 00:53:41,951 INFO [Listener at localhost/37869] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 00:53:42,029 DEBUG [Listener at localhost/37869] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-13 00:53:42,122 INFO [Listener at localhost/37869] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43563 2023-02-13 00:53:42,132 INFO [Listener at localhost/37869] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 00:53:42,134 INFO [Listener at localhost/37869] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 00:53:42,154 INFO [Listener at localhost/37869] zookeeper.RecoverableZooKeeper(93): Process identifier=master:43563 connecting to ZooKeeper ensemble=127.0.0.1:54478 2023-02-13 00:53:42,194 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:435630x0, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 00:53:42,197 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:43563-0x1004b2b47c80000 connected 2023-02-13 00:53:42,225 DEBUG [Listener at localhost/37869] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 00:53:42,226 DEBUG [Listener at localhost/37869] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 00:53:42,229 DEBUG [Listener at localhost/37869] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 00:53:42,237 DEBUG [Listener at localhost/37869] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43563 2023-02-13 00:53:42,237 DEBUG [Listener at localhost/37869] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43563 2023-02-13 00:53:42,237 DEBUG [Listener at localhost/37869] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43563 2023-02-13 00:53:42,238 DEBUG [Listener at localhost/37869] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43563 2023-02-13 00:53:42,238 DEBUG [Listener at localhost/37869] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43563 2023-02-13 00:53:42,244 INFO [Listener at localhost/37869] master.HMaster(439): hbase.rootdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00, hbase.cluster.distributed=false 2023-02-13 00:53:42,318 INFO [Listener at localhost/37869] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=3 2023-02-13 00:53:42,319 INFO [Listener at localhost/37869] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 00:53:42,319 INFO [Listener at localhost/37869] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 00:53:42,319 INFO [Listener at localhost/37869] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 00:53:42,319 INFO [Listener at localhost/37869] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 00:53:42,319 INFO [Listener at localhost/37869] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 00:53:42,324 INFO [Listener at localhost/37869] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 00:53:42,327 INFO [Listener at localhost/37869] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36605 2023-02-13 00:53:42,329 INFO [Listener at localhost/37869] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 00:53:42,336 DEBUG [Listener at localhost/37869] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 00:53:42,336 INFO [Listener at localhost/37869] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 00:53:42,339 INFO [Listener at localhost/37869] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 00:53:42,342 INFO [Listener at localhost/37869] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36605 connecting to ZooKeeper ensemble=127.0.0.1:54478 2023-02-13 00:53:42,354 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:366050x0, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 00:53:42,355 DEBUG [Listener at localhost/37869] zookeeper.ZKUtil(164): regionserver:366050x0, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 00:53:42,360 DEBUG [Listener at localhost/37869] zookeeper.ZKUtil(164): regionserver:366050x0, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 00:53:42,360 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:36605-0x1004b2b47c80001 connected 2023-02-13 00:53:42,361 DEBUG [Listener at localhost/37869] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 00:53:42,380 DEBUG [Listener at localhost/37869] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36605 2023-02-13 00:53:42,385 DEBUG [Listener at localhost/37869] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36605 2023-02-13 00:53:42,395 DEBUG [Listener at localhost/37869] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36605 2023-02-13 00:53:42,398 DEBUG [Listener at localhost/37869] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36605 2023-02-13 00:53:42,398 DEBUG [Listener at localhost/37869] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36605 2023-02-13 00:53:42,404 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,43563,1676249621162 2023-02-13 00:53:42,414 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 00:53:42,415 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,43563,1676249621162 2023-02-13 00:53:42,434 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 00:53:42,434 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 00:53:42,434 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 00:53:42,435 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 00:53:42,436 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,43563,1676249621162 from backup master directory 2023-02-13 00:53:42,436 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 00:53:42,440 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,43563,1676249621162 2023-02-13 00:53:42,440 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 00:53:42,441 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 00:53:42,441 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,43563,1676249621162 2023-02-13 00:53:42,445 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-13 00:53:42,447 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-13 00:53:42,532 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/hbase.id with ID: 7ba88f04-35b8-436e-b12b-88ea71ecab1a 2023-02-13 00:53:42,571 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 00:53:42,585 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 00:53:42,623 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x72b1293e to 127.0.0.1:54478 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:53:42,657 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@635a6705, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:53:42,680 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 00:53:42,682 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-13 00:53:42,700 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-13 00:53:42,701 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-13 00:53:42,702 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-13 00:53:42,706 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-13 00:53:42,707 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 00:53:42,741 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/MasterData/data/master/store-tmp 2023-02-13 00:53:42,769 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:53:42,769 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 00:53:42,770 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 00:53:42,770 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 00:53:42,770 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 00:53:42,770 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 00:53:42,770 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 00:53:42,771 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676249622769Disabling compacts and flushes for region at 1676249622769Disabling writes for close at 1676249622770 (+1 ms)Writing region close event to WAL at 1676249622770Closed at 1676249622770 2023-02-13 00:53:42,772 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/MasterData/WALs/jenkins-hbase4.apache.org,43563,1676249621162 2023-02-13 00:53:42,794 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43563%2C1676249621162, suffix=, logDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/MasterData/WALs/jenkins-hbase4.apache.org,43563,1676249621162, archiveDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/MasterData/oldWALs, maxLogs=10 2023-02-13 00:53:42,852 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36491,DS-bcb7cc95-c419-4075-9477-71577858f0ce,DISK] 2023-02-13 00:53:42,860 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-13 00:53:42,933 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/MasterData/WALs/jenkins-hbase4.apache.org,43563,1676249621162/jenkins-hbase4.apache.org%2C43563%2C1676249621162.1676249622804 2023-02-13 00:53:42,934 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36491,DS-bcb7cc95-c419-4075-9477-71577858f0ce,DISK]] 2023-02-13 00:53:42,934 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:53:42,935 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:53:42,938 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 00:53:42,939 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 00:53:43,019 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-13 00:53:43,027 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-13 00:53:43,055 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-13 00:53:43,067 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:53:43,072 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 00:53:43,074 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 00:53:43,090 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 00:53:43,095 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:53:43,096 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60717885, jitterRate=-0.0952330082654953}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 00:53:43,096 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676249622976Initializing all the Stores at 1676249622979 (+3 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249622979Cleaning up temporary data from old regions at 1676249623081 (+102 ms)Cleaning up detritus from prior splits at 1676249623082 (+1 ms)Region opened successfully at 1676249623096 (+14 ms) 2023-02-13 00:53:43,097 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-13 00:53:43,118 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-02-13 00:53:43,118 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-02-13 00:53:43,120 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:53:43,122 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 00:53:43,161 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 37 msec 2023-02-13 00:53:43,161 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-13 00:53:43,190 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-13 00:53:43,195 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-13 00:53:43,221 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-13 00:53:43,226 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-13 00:53:43,227 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-13 00:53:43,232 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-13 00:53:43,236 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-13 00:53:43,239 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 00:53:43,240 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-13 00:53:43,241 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-13 00:53:43,256 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-13 00:53:43,261 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 00:53:43,261 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 00:53:43,261 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 00:53:43,261 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,43563,1676249621162, sessionid=0x1004b2b47c80000, setting cluster-up flag (Was=false) 2023-02-13 00:53:43,278 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 00:53:43,285 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-13 00:53:43,286 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,43563,1676249621162 2023-02-13 00:53:43,292 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 00:53:43,302 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-13 00:53:43,303 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,43563,1676249621162 2023-02-13 00:53:43,307 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.hbase-snapshot/.tmp 2023-02-13 00:53:43,403 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(952): ClusterId : 7ba88f04-35b8-436e-b12b-88ea71ecab1a 2023-02-13 00:53:43,408 DEBUG [RS:0;jenkins-hbase4:36605] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 00:53:43,419 DEBUG [RS:0;jenkins-hbase4:36605] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 00:53:43,419 DEBUG [RS:0;jenkins-hbase4:36605] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 00:53:43,423 DEBUG [RS:0;jenkins-hbase4:36605] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 00:53:43,424 DEBUG [RS:0;jenkins-hbase4:36605] zookeeper.ReadOnlyZKClient(139): Connect 0x39699047 to 127.0.0.1:54478 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:53:43,429 DEBUG [RS:0;jenkins-hbase4:36605] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@37ae691a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:53:43,429 DEBUG [RS:0;jenkins-hbase4:36605] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3391fb37, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 00:53:43,441 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-13 00:53:43,454 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 00:53:43,454 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 00:53:43,454 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 00:53:43,454 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 00:53:43,454 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-13 00:53:43,455 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:53:43,455 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 00:53:43,455 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:53:43,455 DEBUG [RS:0;jenkins-hbase4:36605] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:36605 2023-02-13 00:53:43,459 INFO [RS:0;jenkins-hbase4:36605] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 00:53:43,459 INFO [RS:0;jenkins-hbase4:36605] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 00:53:43,459 DEBUG [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 00:53:43,459 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249653459 2023-02-13 00:53:43,462 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-13 00:53:43,463 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,43563,1676249621162 with isa=jenkins-hbase4.apache.org/172.31.14.131:36605, startcode=1676249622318 2023-02-13 00:53:43,466 DEBUG [PEWorker-2] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 00:53:43,466 INFO [PEWorker-2] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-13 00:53:43,472 INFO [PEWorker-2] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 00:53:43,473 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-13 00:53:43,480 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-13 00:53:43,480 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-13 00:53:43,481 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-13 00:53:43,481 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-13 00:53:43,482 DEBUG [RS:0;jenkins-hbase4:36605] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 00:53:43,482 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 00:53:43,485 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-13 00:53:43,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-13 00:53:43,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-13 00:53:43,489 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-13 00:53:43,490 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-13 00:53:43,491 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676249623491,5,FailOnTimeoutGroup] 2023-02-13 00:53:43,491 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676249623491,5,FailOnTimeoutGroup] 2023-02-13 00:53:43,491 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 00:53:43,492 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-13 00:53:43,493 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-13 00:53:43,493 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-13 00:53:43,538 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37857, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 00:53:43,549 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:43,564 DEBUG [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00 2023-02-13 00:53:43,564 DEBUG [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:38857 2023-02-13 00:53:43,564 DEBUG [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 00:53:43,569 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 00:53:43,570 DEBUG [RS:0;jenkins-hbase4:36605] zookeeper.ZKUtil(162): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:43,570 WARN [RS:0;jenkins-hbase4:36605] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 00:53:43,571 INFO [RS:0;jenkins-hbase4:36605] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 00:53:43,572 DEBUG [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(1947): logDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/WALs/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:43,573 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,36605,1676249622318] 2023-02-13 00:53:43,581 DEBUG [RS:0;jenkins-hbase4:36605] zookeeper.ZKUtil(162): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:43,592 DEBUG [RS:0;jenkins-hbase4:36605] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 00:53:43,601 INFO [RS:0;jenkins-hbase4:36605] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 00:53:43,627 INFO [RS:0;jenkins-hbase4:36605] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 00:53:43,632 INFO [RS:0;jenkins-hbase4:36605] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 00:53:43,632 INFO [RS:0;jenkins-hbase4:36605] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:53:43,634 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 00:53:43,643 INFO [RS:0;jenkins-hbase4:36605] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-02-13 00:53:43,643 DEBUG [RS:0;jenkins-hbase4:36605] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:53:43,644 DEBUG [RS:0;jenkins-hbase4:36605] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:53:43,644 DEBUG [RS:0;jenkins-hbase4:36605] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:53:43,644 DEBUG [RS:0;jenkins-hbase4:36605] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:53:43,644 DEBUG [RS:0;jenkins-hbase4:36605] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:53:43,645 DEBUG [RS:0;jenkins-hbase4:36605] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 00:53:43,645 DEBUG [RS:0;jenkins-hbase4:36605] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:53:43,645 DEBUG [RS:0;jenkins-hbase4:36605] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:53:43,645 DEBUG [RS:0;jenkins-hbase4:36605] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:53:43,646 DEBUG [RS:0;jenkins-hbase4:36605] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:53:43,647 INFO [RS:0;jenkins-hbase4:36605] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 00:53:43,647 INFO [RS:0;jenkins-hbase4:36605] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 00:53:43,647 INFO [RS:0;jenkins-hbase4:36605] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 00:53:43,663 INFO [RS:0;jenkins-hbase4:36605] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 00:53:43,665 INFO [RS:0;jenkins-hbase4:36605] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36605,1676249622318-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:53:43,683 INFO [RS:0;jenkins-hbase4:36605] regionserver.Replication(203): jenkins-hbase4.apache.org,36605,1676249622318 started 2023-02-13 00:53:43,683 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,36605,1676249622318, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:36605, sessionid=0x1004b2b47c80001 2023-02-13 00:53:43,683 DEBUG [RS:0;jenkins-hbase4:36605] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 00:53:43,683 DEBUG [RS:0;jenkins-hbase4:36605] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:43,683 DEBUG [RS:0;jenkins-hbase4:36605] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36605,1676249622318' 2023-02-13 00:53:43,683 DEBUG [RS:0;jenkins-hbase4:36605] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:53:43,684 DEBUG [RS:0;jenkins-hbase4:36605] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:53:43,685 DEBUG [RS:0;jenkins-hbase4:36605] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 00:53:43,685 DEBUG [RS:0;jenkins-hbase4:36605] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 00:53:43,685 DEBUG [RS:0;jenkins-hbase4:36605] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:43,685 DEBUG [RS:0;jenkins-hbase4:36605] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36605,1676249622318' 2023-02-13 00:53:43,685 DEBUG [RS:0;jenkins-hbase4:36605] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 00:53:43,685 DEBUG [RS:0;jenkins-hbase4:36605] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 00:53:43,686 DEBUG [RS:0;jenkins-hbase4:36605] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 00:53:43,686 INFO [RS:0;jenkins-hbase4:36605] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 00:53:43,686 INFO [RS:0;jenkins-hbase4:36605] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 00:53:43,798 INFO [RS:0;jenkins-hbase4:36605] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36605%2C1676249622318, suffix=, logDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/WALs/jenkins-hbase4.apache.org,36605,1676249622318, archiveDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/oldWALs, maxLogs=32 2023-02-13 00:53:43,817 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36491,DS-bcb7cc95-c419-4075-9477-71577858f0ce,DISK] 2023-02-13 00:53:43,821 INFO [RS:0;jenkins-hbase4:36605] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/WALs/jenkins-hbase4.apache.org,36605,1676249622318/jenkins-hbase4.apache.org%2C36605%2C1676249622318.1676249623801 2023-02-13 00:53:43,821 DEBUG [RS:0;jenkins-hbase4:36605] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36491,DS-bcb7cc95-c419-4075-9477-71577858f0ce,DISK]] 2023-02-13 00:53:43,919 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 00:53:43,921 INFO [PEWorker-2] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 00:53:43,921 INFO [PEWorker-2] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00 2023-02-13 00:53:44,336 DEBUG [PEWorker-2] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:53:44,341 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 00:53:44,345 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740/info 2023-02-13 00:53:44,346 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 00:53:44,347 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:53:44,348 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 00:53:44,351 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740/rep_barrier 2023-02-13 00:53:44,352 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 00:53:44,353 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:53:44,353 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 00:53:44,357 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740/table 2023-02-13 00:53:44,357 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 00:53:44,358 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:53:44,360 DEBUG [PEWorker-2] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740 2023-02-13 00:53:44,361 DEBUG [PEWorker-2] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740 2023-02-13 00:53:44,365 DEBUG [PEWorker-2] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 00:53:44,367 DEBUG [PEWorker-2] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 00:53:44,371 DEBUG [PEWorker-2] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:53:44,373 INFO [PEWorker-2] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61505229, jitterRate=-0.08350066840648651}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 00:53:44,373 DEBUG [PEWorker-2] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676249624336Initializing all the Stores at 1676249624340 (+4 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676249624340Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249624340Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676249624341 (+1 ms)Cleaning up temporary data from old regions at 1676249624363 (+22 ms)Cleaning up detritus from prior splits at 1676249624364 (+1 ms)Region opened successfully at 1676249624373 (+9 ms) 2023-02-13 00:53:44,373 DEBUG [PEWorker-2] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 00:53:44,373 INFO [PEWorker-2] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 00:53:44,373 DEBUG [PEWorker-2] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 00:53:44,373 DEBUG [PEWorker-2] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 00:53:44,373 DEBUG [PEWorker-2] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 00:53:44,374 INFO [PEWorker-2] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 00:53:44,375 DEBUG [PEWorker-2] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676249624373Disabling compacts and flushes for region at 1676249624373Disabling writes for close at 1676249624373Writing region close event to WAL at 1676249624374 (+1 ms)Closed at 1676249624374 2023-02-13 00:53:44,380 DEBUG [PEWorker-2] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 00:53:44,380 INFO [PEWorker-2] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-13 00:53:44,390 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-13 00:53:44,405 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-13 00:53:44,409 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-13 00:53:44,561 DEBUG [jenkins-hbase4:43563] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-02-13 00:53:44,564 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,36605,1676249622318, state=OPENING 2023-02-13 00:53:44,572 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-13 00:53:44,574 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 00:53:44,575 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 00:53:44,577 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:53:44,760 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:44,762 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 00:53:44,766 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44386, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 00:53:44,776 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-13 00:53:44,777 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 00:53:44,780 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36605%2C1676249622318.meta, suffix=.meta, logDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/WALs/jenkins-hbase4.apache.org,36605,1676249622318, archiveDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/oldWALs, maxLogs=32 2023-02-13 00:53:44,796 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36491,DS-bcb7cc95-c419-4075-9477-71577858f0ce,DISK] 2023-02-13 00:53:44,800 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/WALs/jenkins-hbase4.apache.org,36605,1676249622318/jenkins-hbase4.apache.org%2C36605%2C1676249622318.meta.1676249624781.meta 2023-02-13 00:53:44,800 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36491,DS-bcb7cc95-c419-4075-9477-71577858f0ce,DISK]] 2023-02-13 00:53:44,801 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:53:44,803 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-13 00:53:44,823 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-13 00:53:44,828 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-13 00:53:44,834 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-13 00:53:44,834 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:53:44,834 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-13 00:53:44,834 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-13 00:53:44,837 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 00:53:44,839 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740/info 2023-02-13 00:53:44,839 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740/info 2023-02-13 00:53:44,840 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 00:53:44,841 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:53:44,841 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 00:53:44,843 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740/rep_barrier 2023-02-13 00:53:44,843 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740/rep_barrier 2023-02-13 00:53:44,844 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 00:53:44,845 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:53:44,845 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 00:53:44,847 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740/table 2023-02-13 00:53:44,847 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740/table 2023-02-13 00:53:44,847 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 00:53:44,848 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:53:44,850 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740 2023-02-13 00:53:44,853 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740 2023-02-13 00:53:44,857 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 00:53:44,860 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 00:53:44,861 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73687266, jitterRate=0.09802582859992981}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 00:53:44,862 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676249624835Writing region info on filesystem at 1676249624835Initializing all the Stores at 1676249624837 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676249624837Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249624837Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676249624837Cleaning up temporary data from old regions at 1676249624855 (+18 ms)Cleaning up detritus from prior splits at 1676249624856 (+1 ms)Running coprocessor post-open hooks at 1676249624862 (+6 ms)Region opened successfully at 1676249624862 2023-02-13 00:53:44,872 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676249624752 2023-02-13 00:53:44,890 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-13 00:53:44,891 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-13 00:53:44,891 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,36605,1676249622318, state=OPEN 2023-02-13 00:53:44,895 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-13 00:53:44,896 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 00:53:44,899 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-13 00:53:44,900 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,36605,1676249622318 in 318 msec 2023-02-13 00:53:44,904 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-13 00:53:44,904 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 512 msec 2023-02-13 00:53:44,909 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 1.5460 sec 2023-02-13 00:53:44,909 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676249622430, completionTime=-1 2023-02-13 00:53:44,910 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-02-13 00:53:44,910 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:53:44,982 DEBUG [hconnection-0x241edf12-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:53:44,987 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44392, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:53:45,008 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=1 2023-02-13 00:53:45,008 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249685008 2023-02-13 00:53:45,008 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249745008 2023-02-13 00:53:45,009 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 98 msec 2023-02-13 00:53:45,033 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43563,1676249621162-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:53:45,034 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43563,1676249621162-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 00:53:45,034 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43563,1676249621162-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 00:53:45,036 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:43563, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 00:53:45,036 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-13 00:53:45,042 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-13 00:53:45,051 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-13 00:53:45,052 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 00:53:45,062 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-13 00:53:45,064 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 00:53:45,069 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 00:53:45,090 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/hbase/namespace/ca49db847711a2d16c6b58efda0a70df 2023-02-13 00:53:45,093 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/hbase/namespace/ca49db847711a2d16c6b58efda0a70df empty. 2023-02-13 00:53:45,093 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/hbase/namespace/ca49db847711a2d16c6b58efda0a70df 2023-02-13 00:53:45,094 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-13 00:53:45,119 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-13 00:53:45,122 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => ca49db847711a2d16c6b58efda0a70df, NAME => 'hbase:namespace,,1676249625052.ca49db847711a2d16c6b58efda0a70df.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp 2023-02-13 00:53:45,136 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676249625052.ca49db847711a2d16c6b58efda0a70df.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:53:45,136 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing ca49db847711a2d16c6b58efda0a70df, disabling compactions & flushes 2023-02-13 00:53:45,136 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676249625052.ca49db847711a2d16c6b58efda0a70df. 2023-02-13 00:53:45,136 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676249625052.ca49db847711a2d16c6b58efda0a70df. 2023-02-13 00:53:45,136 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676249625052.ca49db847711a2d16c6b58efda0a70df. after waiting 0 ms 2023-02-13 00:53:45,136 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676249625052.ca49db847711a2d16c6b58efda0a70df. 2023-02-13 00:53:45,136 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676249625052.ca49db847711a2d16c6b58efda0a70df. 2023-02-13 00:53:45,137 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for ca49db847711a2d16c6b58efda0a70df: Waiting for close lock at 1676249625136Disabling compacts and flushes for region at 1676249625136Disabling writes for close at 1676249625136Writing region close event to WAL at 1676249625136Closed at 1676249625136 2023-02-13 00:53:45,141 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 00:53:45,161 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676249625052.ca49db847711a2d16c6b58efda0a70df.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676249625144"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249625144"}]},"ts":"1676249625144"} 2023-02-13 00:53:45,192 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 00:53:45,194 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 00:53:45,199 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249625195"}]},"ts":"1676249625195"} 2023-02-13 00:53:45,203 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-13 00:53:45,214 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=ca49db847711a2d16c6b58efda0a70df, ASSIGN}] 2023-02-13 00:53:45,217 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=ca49db847711a2d16c6b58efda0a70df, ASSIGN 2023-02-13 00:53:45,220 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=ca49db847711a2d16c6b58efda0a70df, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=false 2023-02-13 00:53:45,372 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=ca49db847711a2d16c6b58efda0a70df, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:45,372 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676249625052.ca49db847711a2d16c6b58efda0a70df.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676249625371"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249625371"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249625371"}]},"ts":"1676249625371"} 2023-02-13 00:53:45,378 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure ca49db847711a2d16c6b58efda0a70df, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:53:45,538 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676249625052.ca49db847711a2d16c6b58efda0a70df. 2023-02-13 00:53:45,539 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ca49db847711a2d16c6b58efda0a70df, NAME => 'hbase:namespace,,1676249625052.ca49db847711a2d16c6b58efda0a70df.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:53:45,539 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace ca49db847711a2d16c6b58efda0a70df 2023-02-13 00:53:45,539 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676249625052.ca49db847711a2d16c6b58efda0a70df.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:53:45,540 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for ca49db847711a2d16c6b58efda0a70df 2023-02-13 00:53:45,540 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for ca49db847711a2d16c6b58efda0a70df 2023-02-13 00:53:45,542 INFO [StoreOpener-ca49db847711a2d16c6b58efda0a70df-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region ca49db847711a2d16c6b58efda0a70df 2023-02-13 00:53:45,544 DEBUG [StoreOpener-ca49db847711a2d16c6b58efda0a70df-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/namespace/ca49db847711a2d16c6b58efda0a70df/info 2023-02-13 00:53:45,544 DEBUG [StoreOpener-ca49db847711a2d16c6b58efda0a70df-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/namespace/ca49db847711a2d16c6b58efda0a70df/info 2023-02-13 00:53:45,544 INFO [StoreOpener-ca49db847711a2d16c6b58efda0a70df-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ca49db847711a2d16c6b58efda0a70df columnFamilyName info 2023-02-13 00:53:45,545 INFO [StoreOpener-ca49db847711a2d16c6b58efda0a70df-1] regionserver.HStore(310): Store=ca49db847711a2d16c6b58efda0a70df/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:53:45,547 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/namespace/ca49db847711a2d16c6b58efda0a70df 2023-02-13 00:53:45,548 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/namespace/ca49db847711a2d16c6b58efda0a70df 2023-02-13 00:53:45,552 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for ca49db847711a2d16c6b58efda0a70df 2023-02-13 00:53:45,555 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/namespace/ca49db847711a2d16c6b58efda0a70df/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:53:45,556 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened ca49db847711a2d16c6b58efda0a70df; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60872526, jitterRate=-0.0929286777973175}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 00:53:45,557 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for ca49db847711a2d16c6b58efda0a70df: Running coprocessor pre-open hook at 1676249625540Writing region info on filesystem at 1676249625540Initializing all the Stores at 1676249625541 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676249625541Cleaning up temporary data from old regions at 1676249625548 (+7 ms)Cleaning up detritus from prior splits at 1676249625549 (+1 ms)Running coprocessor post-open hooks at 1676249625556 (+7 ms)Region opened successfully at 1676249625556 2023-02-13 00:53:45,558 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676249625052.ca49db847711a2d16c6b58efda0a70df., pid=6, masterSystemTime=1676249625532 2023-02-13 00:53:45,563 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676249625052.ca49db847711a2d16c6b58efda0a70df. 2023-02-13 00:53:45,563 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676249625052.ca49db847711a2d16c6b58efda0a70df. 2023-02-13 00:53:45,565 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=ca49db847711a2d16c6b58efda0a70df, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:45,565 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676249625052.ca49db847711a2d16c6b58efda0a70df.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676249625564"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249625564"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249625564"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249625564"}]},"ts":"1676249625564"} 2023-02-13 00:53:45,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-13 00:53:45,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure ca49db847711a2d16c6b58efda0a70df, server=jenkins-hbase4.apache.org,36605,1676249622318 in 191 msec 2023-02-13 00:53:45,576 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-13 00:53:45,577 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=ca49db847711a2d16c6b58efda0a70df, ASSIGN in 359 msec 2023-02-13 00:53:45,578 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 00:53:45,579 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249625579"}]},"ts":"1676249625579"} 2023-02-13 00:53:45,582 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-13 00:53:45,585 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 00:53:45,588 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 532 msec 2023-02-13 00:53:45,665 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-13 00:53:45,668 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-13 00:53:45,668 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 00:53:45,713 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-13 00:53:45,736 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 00:53:45,742 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 42 msec 2023-02-13 00:53:45,747 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:53:45,764 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 00:53:45,769 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 21 msec 2023-02-13 00:53:45,785 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-13 00:53:45,788 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-13 00:53:45,788 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 3.347sec 2023-02-13 00:53:45,790 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-13 00:53:45,792 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-13 00:53:45,792 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-13 00:53:45,793 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43563,1676249621162-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-13 00:53:45,794 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43563,1676249621162-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-13 00:53:45,806 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-13 00:53:45,811 DEBUG [Listener at localhost/37869] zookeeper.ReadOnlyZKClient(139): Connect 0x0629018b to 127.0.0.1:54478 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:53:45,818 DEBUG [Listener at localhost/37869] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3c1e7f16, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:53:45,835 DEBUG [hconnection-0x77bc4ff1-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:53:45,848 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44402, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:53:45,866 INFO [Listener at localhost/37869] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,43563,1676249621162 2023-02-13 00:53:45,867 DEBUG [Listener at localhost/37869] zookeeper.ReadOnlyZKClient(139): Connect 0x7defec53 to 127.0.0.1:54478 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:53:45,880 DEBUG [Listener at localhost/37869] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@64b944ad, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:53:45,881 INFO [Listener at localhost/37869] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 00:53:45,913 INFO [Listener at localhost/37869] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=211, OpenFileDescriptor=475, MaxFileDescriptor=60000, SystemLoadAverage=284, ProcessCount=168, AvailableMemoryMB=4243 2023-02-13 00:53:45,928 DEBUG [Listener at localhost/37869] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 00:53:45,933 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37070, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 00:53:45,946 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-13 00:53:45,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:45,950 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 00:53:45,953 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 00:53:45,955 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-02-13 00:53:45,956 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 2023-02-13 00:53:45,957 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 empty. 2023-02-13 00:53:45,959 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 2023-02-13 00:53:45,959 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-02-13 00:53:45,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-13 00:53:46,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-13 00:53:46,273 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-13 00:53:46,378 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-02-13 00:53:46,380 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7671): creating {ENCODED => 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp 2023-02-13 00:53:46,391 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:53:46,391 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1603): Closing 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, disabling compactions & flushes 2023-02-13 00:53:46,391 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:46,391 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:46,392 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. after waiting 0 ms 2023-02-13 00:53:46,392 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:46,392 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:46,392 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1557): Region close journal for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0: Waiting for close lock at 1676249626391Disabling compacts and flushes for region at 1676249626391Disabling writes for close at 1676249626392 (+1 ms)Writing region close event to WAL at 1676249626392Closed at 1676249626392 2023-02-13 00:53:46,396 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 00:53:46,397 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676249626397"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249626397"}]},"ts":"1676249626397"} 2023-02-13 00:53:46,400 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 00:53:46,401 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 00:53:46,402 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249626401"}]},"ts":"1676249626401"} 2023-02-13 00:53:46,404 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-02-13 00:53:46,408 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, ASSIGN}] 2023-02-13 00:53:46,410 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, ASSIGN 2023-02-13 00:53:46,411 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=false 2023-02-13 00:53:46,562 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:46,563 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676249626562"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249626562"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249626562"}]},"ts":"1676249626562"} 2023-02-13 00:53:46,566 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:53:46,575 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-13 00:53:46,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:46,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:53:46,727 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 2023-02-13 00:53:46,727 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:53:46,727 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 2023-02-13 00:53:46,727 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 2023-02-13 00:53:46,731 INFO [StoreOpener-9dbd9ef1f4f3ecd6dc6fca04bd7b98f0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 2023-02-13 00:53:46,734 DEBUG [StoreOpener-9dbd9ef1f4f3ecd6dc6fca04bd7b98f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1 2023-02-13 00:53:46,734 DEBUG [StoreOpener-9dbd9ef1f4f3ecd6dc6fca04bd7b98f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1 2023-02-13 00:53:46,738 INFO [StoreOpener-9dbd9ef1f4f3ecd6dc6fca04bd7b98f0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 columnFamilyName family1 2023-02-13 00:53:46,741 INFO [StoreOpener-9dbd9ef1f4f3ecd6dc6fca04bd7b98f0-1] regionserver.HStore(310): Store=9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:53:46,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 2023-02-13 00:53:46,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 2023-02-13 00:53:46,749 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 2023-02-13 00:53:46,765 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:53:46,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59292800, jitterRate=-0.11646842956542969}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 00:53:46,767 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0: Running coprocessor pre-open hook at 1676249626727Writing region info on filesystem at 1676249626727Initializing all the Stores at 1676249626729 (+2 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676249626729Cleaning up temporary data from old regions at 1676249626745 (+16 ms)Cleaning up detritus from prior splits at 1676249626746 (+1 ms)Running coprocessor post-open hooks at 1676249626767 (+21 ms)Region opened successfully at 1676249626767 2023-02-13 00:53:46,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0., pid=11, masterSystemTime=1676249626719 2023-02-13 00:53:46,771 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:46,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:46,772 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:46,773 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676249626772"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249626772"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249626772"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249626772"}]},"ts":"1676249626772"} 2023-02-13 00:53:46,780 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-13 00:53:46,780 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, server=jenkins-hbase4.apache.org,36605,1676249622318 in 210 msec 2023-02-13 00:53:46,784 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-13 00:53:46,784 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, ASSIGN in 372 msec 2023-02-13 00:53:46,786 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 00:53:46,786 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249626786"}]},"ts":"1676249626786"} 2023-02-13 00:53:46,788 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-02-13 00:53:46,792 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 00:53:46,796 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 847 msec 2023-02-13 00:53:47,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-13 00:53:47,077 INFO [Listener at localhost/37869] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-02-13 00:53:47,100 DEBUG [hconnection-0x611a5737-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:53:47,103 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44412, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:53:47,265 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:53:47,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:47,273 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 00:53:47,273 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:53:47,273 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 00:53:47,273 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 00:53:47,274 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:47,274 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:47,276 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:53:47,276 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:47,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:53:47,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:53:47,276 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:47,276 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:53:47,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:47,277 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:47,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:53:47,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:47,278 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:47,279 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:47,280 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 00:53:47,280 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:53:47,280 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 00:53:47,281 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:53:47,281 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:53:47,281 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:47,282 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. started... 2023-02-13 00:53:47,283 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-13 00:53:47,840 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130ff87fe5f6b804db3854f9702855c2759 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130ff87fe5f6b804db3854f9702855c2759 2023-02-13 00:53:48,258 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/.tmp/family1/9749a7005e144eb7997753027f825d16 2023-02-13 00:53:48,269 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/.tmp/family1/9749a7005e144eb7997753027f825d16 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/9749a7005e144eb7997753027f825d16 2023-02-13 00:53:48,278 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/9749a7005e144eb7997753027f825d16, entries=1000, sequenceid=5, filesize=175.8 K 2023-02-13 00:53:48,281 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 in 998ms, sequenceid=5, compaction requested=false 2023-02-13 00:53:48,283 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 00:53:48,286 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0: 2023-02-13 00:53:48,286 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:48,286 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:53:48,287 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:53:48,287 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:48,287 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 00:53:48,287 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 00:53:48,289 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:48,290 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:48,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:48,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:48,290 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,290 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:53:48,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:48,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:48,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:48,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:48,292 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 00:53:48,292 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 00:53:48,292 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1eb5f7f7[Count = 0] remaining members to acquire global barrier 2023-02-13 00:53:48,293 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,294 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,295 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 00:53:48,295 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 00:53:48,295 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:53:48,295 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:48,295 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:53:48,298 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:48,298 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 00:53:48,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:48,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:48,298 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:53:48,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:48,298 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 00:53:48,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:48,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:48,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:48,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:48,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:48,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:53:48,302 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 00:53:48,302 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:53:48,302 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 00:53:48,302 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:53:48,302 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,302 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:53:48,306 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,306 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:48,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:48,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,306 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:53:48,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,307 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:48,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:48,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:53:48,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:53:48,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:48,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,309 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:48,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:48,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:48,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:48,319 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:48,319 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:53:48,319 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:53:48,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:53:48,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:53:48,319 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,319 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,319 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:48,319 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,319 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:53:48,319 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 00:53:48,319 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,320 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:53:48,320 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:53:48,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:53:48,322 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 00:53:48,322 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:53:48,423 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:53:48,428 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:53:48,440 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:53:48,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 00:53:48,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,443 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 00:53:48,443 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:53:48,443 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 00:53:48,444 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 00:53:48,444 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,444 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,450 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:48,450 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:53:48,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:53:48,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:53:48,450 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:48,450 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:53:48,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,451 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:53:48,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,452 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,452 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 00:53:48,453 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:48,453 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 00:53:48,453 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:53:48,453 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 00:53:48,454 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:53:48,454 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:53:48,454 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:48,454 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. started... 2023-02-13 00:53:48,454 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-13 00:53:48,884 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128304da1ac688644c2ad5b4afd85c8813a to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128304da1ac688644c2ad5b4afd85c8813a 2023-02-13 00:53:49,295 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/.tmp/family1/ce9a796f97f94c94b85773892e886ba5 2023-02-13 00:53:49,305 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/.tmp/family1/ce9a796f97f94c94b85773892e886ba5 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/ce9a796f97f94c94b85773892e886ba5 2023-02-13 00:53:49,312 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/ce9a796f97f94c94b85773892e886ba5, entries=1, sequenceid=9, filesize=5.0 K 2023-02-13 00:53:49,314 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 in 860ms, sequenceid=9, compaction requested=false 2023-02-13 00:53:49,314 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0: 2023-02-13 00:53:49,314 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:49,314 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:53:49,315 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:53:49,315 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:49,315 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 00:53:49,315 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 00:53:49,317 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:49,317 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:49,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:49,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:49,318 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,318 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:53:49,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:49,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:49,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:49,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:49,320 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 00:53:49,320 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@67c4ad9[Count = 0] remaining members to acquire global barrier 2023-02-13 00:53:49,320 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 00:53:49,320 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,323 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,323 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 00:53:49,323 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 00:53:49,323 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:53:49,323 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:49,323 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:53:49,325 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 00:53:49,325 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:49,325 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:53:49,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:49,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:49,325 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 00:53:49,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:49,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:49,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:49,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:49,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:49,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:49,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:53:49,329 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 00:53:49,329 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 00:53:49,329 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:53:49,329 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:53:49,329 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,329 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:53:49,333 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,333 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:49,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:49,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,334 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:53:49,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:53:49,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:49,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:53:49,334 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:49,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:49,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:49,336 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:49,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:49,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:49,341 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:49,341 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:53:49,341 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:53:49,341 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,342 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:53:49,341 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:53:49,342 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 00:53:49,342 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:53:49,342 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:49,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:53:49,342 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,342 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,342 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,342 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 00:53:49,343 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:53:49,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:53:49,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:53:49,443 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:53:49,444 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:53:49,452 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:53:49,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 00:53:49,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,455 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 00:53:49,455 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:53:49,455 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 00:53:49,455 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 00:53:49,456 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,456 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,457 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:53:49,457 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:49,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:53:49,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:53:49,458 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:49,458 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:53:49,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,459 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:53:49,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,459 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,460 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 00:53:49,460 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:49,460 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 00:53:49,460 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:53:49,460 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 00:53:49,460 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:53:49,460 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:53:49,460 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:49,460 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. started... 2023-02-13 00:53:49,461 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:53:49,597 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-13 00:53:49,599 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-13 00:53:49,897 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120165d7708818b24c55a55bcdbdffe351a5 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120165d7708818b24c55a55bcdbdffe351a5 2023-02-13 00:53:50,309 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/.tmp/family1/a78b4ad8109c40c9a02b8ab654e3e35a 2023-02-13 00:53:50,318 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/.tmp/family1/a78b4ad8109c40c9a02b8ab654e3e35a as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/a78b4ad8109c40c9a02b8ab654e3e35a 2023-02-13 00:53:50,326 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/a78b4ad8109c40c9a02b8ab654e3e35a, entries=1, sequenceid=13, filesize=5.0 K 2023-02-13 00:53:50,327 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 in 866ms, sequenceid=13, compaction requested=true 2023-02-13 00:53:50,328 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0: 2023-02-13 00:53:50,328 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:50,328 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:53:50,328 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:53:50,328 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:50,328 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 00:53:50,328 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 00:53:50,330 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:50,330 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:50,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:50,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:50,331 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:50,331 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:53:50,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:50,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:50,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:50,333 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 00:53:50,333 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@79c62583[Count = 0] remaining members to acquire global barrier 2023-02-13 00:53:50,333 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 00:53:50,333 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,334 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,334 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 00:53:50,334 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 00:53:50,334 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:53:50,334 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:50,334 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:53:50,337 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:50,337 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 00:53:50,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:50,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:50,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:50,337 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:53:50,338 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 00:53:50,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:50,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:50,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:50,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:50,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:50,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:53:50,341 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 00:53:50,341 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 00:53:50,341 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:53:50,341 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:53:50,341 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,341 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:53:50,343 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,343 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:50,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:50,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,343 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:53:50,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:50,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:53:50,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:53:50,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,348 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:50,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:50,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:50,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:50,350 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:50,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:50,354 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:50,354 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,354 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:53:50,354 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:53:50,354 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 00:53:50,354 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:53:50,354 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:50,354 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:53:50,355 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,354 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:53:50,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:53:50,355 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,355 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,355 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 00:53:50,356 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:53:50,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:53:50,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:53:50,393 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 00:53:50,456 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:53:50,457 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:53:50,466 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:53:50,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 00:53:50,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,469 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 00:53:50,469 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:53:50,470 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 00:53:50,470 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 00:53:50,470 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,470 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,473 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:53:50,473 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:50,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:53:50,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:53:50,473 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:50,473 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:53:50,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,474 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:53:50,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,474 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,475 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 00:53:50,475 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:50,475 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 00:53:50,475 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:53:50,475 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 00:53:50,476 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:53:50,476 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:53:50,476 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:50,476 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. started... 2023-02-13 00:53:50,476 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:53:50,909 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205db706bc5b03344d1a1c57e4cb4621409 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205db706bc5b03344d1a1c57e4cb4621409 2023-02-13 00:53:51,321 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/.tmp/family1/798f01a078ac4506800dc08905d987e6 2023-02-13 00:53:51,322 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 00:53:51,323 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-13 00:53:51,331 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/.tmp/family1/798f01a078ac4506800dc08905d987e6 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/798f01a078ac4506800dc08905d987e6 2023-02-13 00:53:51,343 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/798f01a078ac4506800dc08905d987e6, entries=1, sequenceid=17, filesize=5.0 K 2023-02-13 00:53:51,344 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 in 868ms, sequenceid=17, compaction requested=true 2023-02-13 00:53:51,344 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0: 2023-02-13 00:53:51,344 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:51,344 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:53:51,345 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:53:51,345 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:51,345 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 00:53:51,345 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 00:53:51,347 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:51,347 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:51,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:51,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:51,347 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,348 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:53:51,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:51,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:51,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:51,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:51,349 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 00:53:51,350 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2032330f[Count = 0] remaining members to acquire global barrier 2023-02-13 00:53:51,350 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 00:53:51,350 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,351 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,351 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 00:53:51,351 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 00:53:51,351 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:53:51,351 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:51,351 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:53:51,353 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:51,353 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 00:53:51,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:51,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:51,353 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:53:51,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:51,353 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 00:53:51,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:51,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:51,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:51,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:51,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:51,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:53:51,357 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 00:53:51,357 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 00:53:51,357 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:53:51,357 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:53:51,357 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,357 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:53:51,361 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,361 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:51,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:51,361 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:53:51,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:51,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:53:51,362 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:51,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:53:51,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:51,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:51,363 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:51,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:51,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:51,370 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:51,370 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:53:51,370 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:53:51,370 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:53:51,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:53:51,370 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 00:53:51,370 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:53:51,370 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,370 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:53:51,370 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:51,370 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,370 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,370 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,371 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 00:53:51,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:53:51,371 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:53:51,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:53:51,471 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:53:51,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:53:51,480 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:53:51,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 00:53:51,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,482 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 00:53:51,482 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:53:51,483 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 00:53:51,483 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 00:53:51,483 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,483 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,485 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:51,485 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:53:51,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:53:51,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:53:51,485 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:51,485 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:53:51,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,486 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:53:51,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,486 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,487 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 00:53:51,487 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:51,487 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 00:53:51,487 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:53:51,487 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 00:53:51,487 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:53:51,487 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:53:51,487 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:51,488 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. started... 2023-02-13 00:53:51,488 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:53:51,911 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122891e54be4a71f4c9da1241c53aa7ee731 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015122891e54be4a71f4c9da1241c53aa7ee731 2023-02-13 00:53:52,325 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/.tmp/family1/70c2358394354f3fb2eef7105860f6d0 2023-02-13 00:53:52,334 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/.tmp/family1/70c2358394354f3fb2eef7105860f6d0 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/70c2358394354f3fb2eef7105860f6d0 2023-02-13 00:53:52,342 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/70c2358394354f3fb2eef7105860f6d0, entries=1, sequenceid=21, filesize=5.0 K 2023-02-13 00:53:52,343 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 in 855ms, sequenceid=21, compaction requested=true 2023-02-13 00:53:52,343 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0: 2023-02-13 00:53:52,343 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:52,344 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:53:52,344 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:53:52,344 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,344 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 00:53:52,344 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 00:53:52,350 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,350 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:52,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:52,350 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,350 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:53:52,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:52,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:52,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:52,352 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 00:53:52,352 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@118972b1[Count = 0] remaining members to acquire global barrier 2023-02-13 00:53:52,352 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 00:53:52,352 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,354 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,354 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 00:53:52,354 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 00:53:52,354 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:53:52,354 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,354 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:53:52,356 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 00:53:52,356 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,356 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:53:52,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:52,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:52,356 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 00:53:52,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:52,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:52,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:52,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:53:52,359 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 00:53:52,359 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 00:53:52,359 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:53:52,359 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:53:52,359 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,359 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:53:52,361 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,361 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,361 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:53:52,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:52,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:52,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:53:52,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:53:52,362 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:52,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:52,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,363 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:52,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,367 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,367 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:53:52,367 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,367 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:53:52,367 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 00:53:52,367 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:53:52,367 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:53:52,367 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:53:52,368 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:53:52,368 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-13 00:53:52,368 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,368 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 00:53:52,368 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,368 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:53:52,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:53:52,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:53:52,368 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,468 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:53:52,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:53:52,477 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:53:52,479 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 00:53:52,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,480 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 00:53:52,480 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:53:52,480 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 00:53:52,480 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 00:53:52,481 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,481 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,482 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:53:52,482 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:53:52,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:53:52,483 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,483 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:53:52,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,483 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:53:52,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,484 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,484 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 00:53:52,484 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,484 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 00:53:52,484 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:53:52,485 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 00:53:52,485 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:53:52,485 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:53:52,485 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:52,485 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. started... 2023-02-13 00:53:52,485 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:53:52,507 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231d15f5898b9c6401897bed7aa1d65f40a to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231d15f5898b9c6401897bed7aa1d65f40a 2023-02-13 00:53:52,915 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/.tmp/family1/6b2da2e6e66544caa1d6ce53c229c22b 2023-02-13 00:53:52,926 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/.tmp/family1/6b2da2e6e66544caa1d6ce53c229c22b as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/6b2da2e6e66544caa1d6ce53c229c22b 2023-02-13 00:53:52,934 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/6b2da2e6e66544caa1d6ce53c229c22b, entries=1, sequenceid=25, filesize=5.0 K 2023-02-13 00:53:52,936 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 in 451ms, sequenceid=25, compaction requested=true 2023-02-13 00:53:52,936 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0: 2023-02-13 00:53:52,937 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:52,937 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:53:52,937 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:53:52,937 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,937 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 00:53:52,937 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 00:53:52,939 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,939 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:52,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:52,940 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,940 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:53:52,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:52,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:52,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:52,942 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 00:53:52,942 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7547c8db[Count = 0] remaining members to acquire global barrier 2023-02-13 00:53:52,942 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 00:53:52,942 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,943 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,943 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 00:53:52,943 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 00:53:52,943 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:53:52,944 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,944 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:53:52,946 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:52,946 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 00:53:52,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:52,946 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:53:52,946 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 00:53:52,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:52,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:52,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:52,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:53:52,949 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 00:53:52,949 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 00:53:52,950 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:53:52,950 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:53:52,950 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,950 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:53:52,951 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,951 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,951 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:53:52,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:52,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:52,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:53:52,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:53:52,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:52,952 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:52,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,954 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:52,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,956 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,956 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:53:52,956 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:53:52,956 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,956 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-13 00:53:52,956 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:53:52,956 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:52,956 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 00:53:52,957 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,956 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:53:52,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:53:52,957 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,957 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:53:52,957 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:52,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:53:52,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:53:52,958 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 00:53:52,958 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:53:53,058 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:53:53,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:53:53,067 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:53:53,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 00:53:53,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,070 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 00:53:53,070 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:53:53,071 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 00:53:53,071 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 00:53:53,071 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,071 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,072 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:53:53,072 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:53,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:53:53,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:53:53,073 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:53,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,073 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:53:53,073 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:53:53,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,074 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,075 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 00:53:53,075 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,075 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 00:53:53,075 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:53:53,075 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 00:53:53,075 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:53:53,075 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:53:53,075 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:53,076 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. started... 2023-02-13 00:53:53,076 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:53:53,497 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010149c0b4d0513640bda8c06eb397606871 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010149c0b4d0513640bda8c06eb397606871 2023-02-13 00:53:53,907 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/.tmp/family1/76dbb071517e4a4185a9623d24677031 2023-02-13 00:53:53,917 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/.tmp/family1/76dbb071517e4a4185a9623d24677031 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/76dbb071517e4a4185a9623d24677031 2023-02-13 00:53:53,926 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/76dbb071517e4a4185a9623d24677031, entries=1, sequenceid=29, filesize=5.0 K 2023-02-13 00:53:53,927 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 in 851ms, sequenceid=29, compaction requested=true 2023-02-13 00:53:53,927 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0: 2023-02-13 00:53:53,927 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:53,927 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:53:53,928 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:53:53,928 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:53,928 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 00:53:53,928 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 00:53:53,930 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:53,931 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:53,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:53,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:53,931 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,931 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:53:53,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:53,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:53,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:53,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:53,933 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 00:53:53,933 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3de511cd[Count = 0] remaining members to acquire global barrier 2023-02-13 00:53:53,933 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 00:53:53,933 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,934 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,935 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 00:53:53,935 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 00:53:53,935 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:53:53,935 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:53,935 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:53:53,937 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 00:53:53,938 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:53,938 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:53:53,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:53,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:53,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:53,938 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 00:53:53,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:53,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:53,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:53,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:53,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:53,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:53:53,941 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 00:53:53,941 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 00:53:53,941 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:53:53,941 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:53:53,941 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,941 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:53:53,943 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,943 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:53,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:53,943 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:53:53,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:53,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:53:53,944 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:53,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:53:53,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:53,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,945 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:53,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:53,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:53,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:53,951 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:53,951 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,951 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:53:53,951 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:53:53,951 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:53:53,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:53:53,951 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-13 00:53:53,951 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 00:53:53,951 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:53:53,951 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:53,951 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:53:53,952 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,952 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,952 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:53,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:53:53,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:53:53,952 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 00:53:53,953 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:53:54,053 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:53:54,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:53:54,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:53:54,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 00:53:54,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,064 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 00:53:54,064 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:53:54,065 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 00:53:54,065 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 00:53:54,065 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,065 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,068 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:54,068 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:53:54,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:53:54,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:53:54,068 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:54,068 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:53:54,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,069 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:53:54,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,069 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,070 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 00:53:54,070 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,070 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 00:53:54,070 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:53:54,070 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 00:53:54,070 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:53:54,070 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:53:54,070 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:54,071 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. started... 2023-02-13 00:53:54,071 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:53:54,091 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103b38be679e5874a988c8bae0fd7067b30 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103b38be679e5874a988c8bae0fd7067b30 2023-02-13 00:53:54,501 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/.tmp/family1/6e8aae8946dd4b6a9d980ed193e9fe1b 2023-02-13 00:53:54,509 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/.tmp/family1/6e8aae8946dd4b6a9d980ed193e9fe1b as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/6e8aae8946dd4b6a9d980ed193e9fe1b 2023-02-13 00:53:54,517 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/6e8aae8946dd4b6a9d980ed193e9fe1b, entries=1, sequenceid=33, filesize=5.0 K 2023-02-13 00:53:54,519 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 in 448ms, sequenceid=33, compaction requested=true 2023-02-13 00:53:54,519 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0: 2023-02-13 00:53:54,519 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:54,519 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:53:54,519 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:53:54,519 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:54,519 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 00:53:54,520 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 00:53:54,521 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,521 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:54,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:54,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:54,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:54,522 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,522 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:53:54,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:54,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:54,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:54,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:54,523 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 00:53:54,524 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@cfef735[Count = 0] remaining members to acquire global barrier 2023-02-13 00:53:54,524 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 00:53:54,524 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,525 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,525 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:54,526 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:53:54,526 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 00:53:54,526 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 00:53:54,526 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:53:54,527 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 00:53:54,527 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:54,527 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:53:54,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:54,527 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 00:53:54,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:54,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:54,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:54,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:54,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:54,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:54,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:54,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:53:54,531 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 00:53:54,531 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 00:53:54,531 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:53:54,531 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:53:54,531 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,531 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:53:54,534 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,534 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:54,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:54,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:54,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,534 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:53:54,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:54,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:53:54,543 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:54,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:53:54,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:54,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:54,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:54,544 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:54,547 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:54,547 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:53:54,547 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:53:54,547 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,547 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 00:53:54,547 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:53:54,547 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:53:54,547 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:53:54,547 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:54,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:53:54,548 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,548 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,548 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 00:53:54,548 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,548 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:53:54,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:53:54,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:53:54,649 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:53:54,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:53:54,658 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:53:54,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 00:53:54,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,660 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 00:53:54,660 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:53:54,661 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 00:53:54,661 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 00:53:54,661 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,661 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,663 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:53:54,663 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:54,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:53:54,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:53:54,663 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:54,663 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:53:54,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,664 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:53:54,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,664 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,665 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 00:53:54,665 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:54,666 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 00:53:54,666 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:53:54,667 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 00:53:54,667 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:53:54,667 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:53:54,667 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:54,667 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. started... 2023-02-13 00:53:54,667 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:53:55,091 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509076eed08be3a0f4626b9b202916167a989 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201509076eed08be3a0f4626b9b202916167a989 2023-02-13 00:53:55,511 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/.tmp/family1/174c41906bb641ec996ee4c25a5e86d3 2023-02-13 00:53:55,521 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/.tmp/family1/174c41906bb641ec996ee4c25a5e86d3 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/174c41906bb641ec996ee4c25a5e86d3 2023-02-13 00:53:55,529 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/174c41906bb641ec996ee4c25a5e86d3, entries=1, sequenceid=37, filesize=5.0 K 2023-02-13 00:53:55,530 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 in 863ms, sequenceid=37, compaction requested=true 2023-02-13 00:53:55,531 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0: 2023-02-13 00:53:55,531 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:55,531 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:53:55,531 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:53:55,531 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:55,531 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 00:53:55,531 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 00:53:55,533 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:55,533 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:55,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:55,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:55,534 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,534 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:53:55,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:55,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:55,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:55,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:55,536 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 00:53:55,536 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@563800e1[Count = 0] remaining members to acquire global barrier 2023-02-13 00:53:55,536 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 00:53:55,536 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,537 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,537 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 00:53:55,538 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 00:53:55,538 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:53:55,538 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:55,538 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:53:55,539 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 00:53:55,539 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:55,539 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:53:55,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:55,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:55,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:55,539 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 00:53:55,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:55,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:55,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:55,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:55,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:55,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:53:55,543 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 00:53:55,543 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 00:53:55,543 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:53:55,543 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:53:55,543 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,543 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:53:55,545 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,545 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:55,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:55,545 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:53:55,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,546 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:55,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:55,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:53:55,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:53:55,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:55,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,547 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:55,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:55,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:55,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:55,550 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:55,550 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,550 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:53:55,550 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:53:55,550 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:55,550 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:53:55,550 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-13 00:53:55,550 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 00:53:55,550 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:53:55,550 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:53:55,550 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,550 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:53:55,551 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,551 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 00:53:55,551 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:53:55,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:53:55,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:53:55,651 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:53:55,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:53:55,661 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:53:55,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 00:53:55,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,663 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 00:53:55,663 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:53:55,664 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 00:53:55,664 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 00:53:55,665 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,665 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,666 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:53:55,666 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:55,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:53:55,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:53:55,666 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:55,666 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:53:55,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,667 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:53:55,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,668 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,672 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 00:53:55,672 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:55,672 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 00:53:55,672 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:53:55,673 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 00:53:55,673 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:53:55,673 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:53:55,673 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:55,674 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. started... 2023-02-13 00:53:55,674 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:53:56,097 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120ab6c17e2149e4850957d8a61f86bebda to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120ab6c17e2149e4850957d8a61f86bebda 2023-02-13 00:53:56,541 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/.tmp/family1/511bd27d9f3a4e069143c1caa035ff94 2023-02-13 00:53:56,549 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/.tmp/family1/511bd27d9f3a4e069143c1caa035ff94 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/511bd27d9f3a4e069143c1caa035ff94 2023-02-13 00:53:56,556 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/511bd27d9f3a4e069143c1caa035ff94, entries=1, sequenceid=41, filesize=5.0 K 2023-02-13 00:53:56,557 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 in 883ms, sequenceid=41, compaction requested=true 2023-02-13 00:53:56,557 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0: 2023-02-13 00:53:56,557 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:56,557 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:53:56,558 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:53:56,558 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:56,558 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 00:53:56,558 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 00:53:56,560 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:56,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:56,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:56,560 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:56,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:56,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:56,560 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:56,561 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:53:56,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:56,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:56,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:56,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:56,562 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 00:53:56,562 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1d721bed[Count = 0] remaining members to acquire global barrier 2023-02-13 00:53:56,562 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 00:53:56,562 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:56,564 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:56,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:56,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:56,564 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 00:53:56,564 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 00:53:56,564 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:53:56,565 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:56,565 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:53:56,570 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 00:53:56,570 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:56,570 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:53:56,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:56,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:56,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:56,570 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 00:53:56,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:56,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:56,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:56,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:56,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:56,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:56,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:56,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:53:56,574 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 00:53:56,574 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 00:53:56,575 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:53:56,575 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:53:56,575 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:56,575 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:53:56,577 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:56,577 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:56,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:56,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:56,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:53:56,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:53:56,577 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:53:56,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:56,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:53:56,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:53:56,577 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:56,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:53:56,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:56,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:56,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:53:56,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:56,579 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:56,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:56,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:53:56,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:56,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:56,581 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:53:56,581 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:56,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:53:56,582 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:56,582 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:53:56,582 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:53:56,582 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:56,582 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 00:53:56,582 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:56,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:53:56,582 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:56,582 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:53:56,582 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:56,582 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:56,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:53:56,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:53:56,583 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 00:53:56,583 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:53:56,683 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:53:56,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:53:56,689 DEBUG [Listener at localhost/37869] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 00:53:56,692 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42064, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 00:53:56,693 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43563] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:56,708 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-13 00:53:56,708 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-13 00:53:56,710 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:53:56,710 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676250236710 2023-02-13 00:53:56,714 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 00:53:56,718 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-02-13 00:53:56,718 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 00:53:56,718 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] zookeeper.ReadOnlyZKClient(139): Connect 0x10c3ee4b to 127.0.0.1:54478 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:53:56,722 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1e7751aa, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:53:56,724 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 00:53:57,208 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103c0d580d3b8b64bfca3a7df9be7326c2c to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103c0d580d3b8b64bfca3a7df9be7326c2c 2023-02-13 00:53:57,210 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 00:53:57,228 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:53:57,230 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41802, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:53:57,241 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 00:53:57,243 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42072, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 00:53:57,285 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/2d114c4015224a92b6841e73dc314963 first=Optional[r3] last=Optional[r6] 2023-02-13 00:53:57,297 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/2d114c4015224a92b6841e73dc314963}] 2023-02-13 00:53:57,314 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/2d114c4015224a92b6841e73dc314963 for inclusion in 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1 2023-02-13 00:53:57,321 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-13 00:53:57,321 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 00:53:57,321 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0: 2023-02-13 00:53:57,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/2d114c4015224a92b6841e73dc314963 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__us23606qvaoaqdjf26m6cj5k1noq3md0gfmsrp99b62vd6rjpa8cevnalvsio1j6/family1/2d114c4015224a92b6841e73dc314963 2023-02-13 00:53:57,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__us23606qvaoaqdjf26m6cj5k1noq3md0gfmsrp99b62vd6rjpa8cevnalvsio1j6/family1/2d114c4015224a92b6841e73dc314963 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/93792048ab754053b8af3679a57cebf2_SeqId_44_ 2023-02-13 00:53:57,327 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__us23606qvaoaqdjf26m6cj5k1noq3md0gfmsrp99b62vd6rjpa8cevnalvsio1j6/family1/2d114c4015224a92b6841e73dc314963 into 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/93792048ab754053b8af3679a57cebf2_SeqId_44_ - updating store file list. 2023-02-13 00:53:57,332 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/93792048ab754053b8af3679a57cebf2_SeqId_44_ into 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1 2023-02-13 00:53:57,332 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__us23606qvaoaqdjf26m6cj5k1noq3md0gfmsrp99b62vd6rjpa8cevnalvsio1j6/family1/2d114c4015224a92b6841e73dc314963 into 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/93792048ab754053b8af3679a57cebf2_SeqId_44_) 2023-02-13 00:53:57,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__us23606qvaoaqdjf26m6cj5k1noq3md0gfmsrp99b62vd6rjpa8cevnalvsio1j6/family1/2d114c4015224a92b6841e73dc314963 2023-02-13 00:53:57,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-13 00:53:57,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-13 00:53:57,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(1912): 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1 is initiating minor compaction 2023-02-13 00:53:57,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@64b352a0; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:53:57,344 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:57,345 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/ce9a796f97f94c94b85773892e886ba5, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/a78b4ad8109c40c9a02b8ab654e3e35a, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/798f01a078ac4506800dc08905d987e6, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/70c2358394354f3fb2eef7105860f6d0, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/6b2da2e6e66544caa1d6ce53c229c22b, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/76dbb071517e4a4185a9623d24677031, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/6e8aae8946dd4b6a9d980ed193e9fe1b, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/174c41906bb641ec996ee4c25a5e86d3, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/511bd27d9f3a4e069143c1caa035ff94, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/93792048ab754053b8af3679a57cebf2_SeqId_44_] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/.tmp, totalSize=50.2 K 2023-02-13 00:53:57,345 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting ce9a796f97f94c94b85773892e886ba5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-13 00:53:57,346 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting a78b4ad8109c40c9a02b8ab654e3e35a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-13 00:53:57,346 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 798f01a078ac4506800dc08905d987e6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-13 00:53:57,346 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 70c2358394354f3fb2eef7105860f6d0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-13 00:53:57,346 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 6b2da2e6e66544caa1d6ce53c229c22b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-13 00:53:57,346 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 76dbb071517e4a4185a9623d24677031, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-13 00:53:57,346 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 6e8aae8946dd4b6a9d980ed193e9fe1b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-13 00:53:57,346 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 174c41906bb641ec996ee4c25a5e86d3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-13 00:53:57,346 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 511bd27d9f3a4e069143c1caa035ff94, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-13 00:53:57,346 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 93792048ab754053b8af3679a57cebf2_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-13 00:53:57,358 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 00:53:57,360 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:53:57,366 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015122891e54be4a71f4c9da1241c53aa7ee731 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015122891e54be4a71f4c9da1241c53aa7ee731 2023-02-13 00:53:57,367 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231d15f5898b9c6401897bed7aa1d65f40a to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231d15f5898b9c6401897bed7aa1d65f40a 2023-02-13 00:53:57,370 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010149c0b4d0513640bda8c06eb397606871 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010149c0b4d0513640bda8c06eb397606871 2023-02-13 00:53:57,372 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103b38be679e5874a988c8bae0fd7067b30 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103b38be679e5874a988c8bae0fd7067b30 2023-02-13 00:53:57,372 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-02-13 00:53:57,372 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-13 00:53:57,373 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 00:53:57,373 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 00:53:57,373 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 00:53:57,373 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 00:53:57,373 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 00:53:57,373 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 00:53:57,373 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 00:53:57,373 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 00:53:57,401 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0#family1#compaction#10 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:53:58,227 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/.tmp/family1/222c3a8ac46046798650e8ea345fa89d as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/222c3a8ac46046798650e8ea345fa89d 2023-02-13 00:53:58,245 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1 of 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 into 222c3a8ac46046798650e8ea345fa89d(size=6.6 K), total size for store is 182.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:53:58,245 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0: 2023-02-13 00:53:58,251 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1676249637342; duration=0sec 2023-02-13 00:53:58,253 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:53:58,257 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/ce9a796f97f94c94b85773892e886ba5, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/a78b4ad8109c40c9a02b8ab654e3e35a, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/798f01a078ac4506800dc08905d987e6, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/70c2358394354f3fb2eef7105860f6d0, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/6b2da2e6e66544caa1d6ce53c229c22b, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/76dbb071517e4a4185a9623d24677031, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/6e8aae8946dd4b6a9d980ed193e9fe1b, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/174c41906bb641ec996ee4c25a5e86d3, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/511bd27d9f3a4e069143c1caa035ff94, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/93792048ab754053b8af3679a57cebf2_SeqId_44_] to archive 2023-02-13 00:53:58,258 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:53:58,261 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/ce9a796f97f94c94b85773892e886ba5 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/ce9a796f97f94c94b85773892e886ba5 2023-02-13 00:53:58,262 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/a78b4ad8109c40c9a02b8ab654e3e35a to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/a78b4ad8109c40c9a02b8ab654e3e35a 2023-02-13 00:53:58,264 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512056a7b98b8eb70441393a24a2cd0d2a91d to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512056a7b98b8eb70441393a24a2cd0d2a91d 2023-02-13 00:53:58,265 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/798f01a078ac4506800dc08905d987e6 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/798f01a078ac4506800dc08905d987e6 2023-02-13 00:53:58,266 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 00:53:58,267 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/70c2358394354f3fb2eef7105860f6d0 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/70c2358394354f3fb2eef7105860f6d0 2023-02-13 00:53:58,269 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/6b2da2e6e66544caa1d6ce53c229c22b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/6b2da2e6e66544caa1d6ce53c229c22b 2023-02-13 00:53:58,271 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/76dbb071517e4a4185a9623d24677031 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/76dbb071517e4a4185a9623d24677031 2023-02-13 00:53:58,272 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/6e8aae8946dd4b6a9d980ed193e9fe1b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/6e8aae8946dd4b6a9d980ed193e9fe1b 2023-02-13 00:53:58,274 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/174c41906bb641ec996ee4c25a5e86d3 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/174c41906bb641ec996ee4c25a5e86d3 2023-02-13 00:53:58,275 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/511bd27d9f3a4e069143c1caa035ff94 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/511bd27d9f3a4e069143c1caa035ff94 2023-02-13 00:53:58,277 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/93792048ab754053b8af3679a57cebf2_SeqId_44_ to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/93792048ab754053b8af3679a57cebf2_SeqId_44_ 2023-02-13 00:53:58,300 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/80938da918ea47aab571b657b18c1ab7 first=Optional[r00] last=Optional[r2] 2023-02-13 00:53:58,301 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/80938da918ea47aab571b657b18c1ab7}] 2023-02-13 00:53:58,304 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/80938da918ea47aab571b657b18c1ab7 for inclusion in 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1 2023-02-13 00:53:58,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-13 00:53:58,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 00:53:58,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0: 2023-02-13 00:53:58,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/80938da918ea47aab571b657b18c1ab7 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__o4q7tc7asepj5u8rl71cc9dsch6mfcr7athn20oc15ul5r72qrfjql8e640964ud/family1/80938da918ea47aab571b657b18c1ab7 2023-02-13 00:53:58,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__o4q7tc7asepj5u8rl71cc9dsch6mfcr7athn20oc15ul5r72qrfjql8e640964ud/family1/80938da918ea47aab571b657b18c1ab7 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/75413152bd504844b9f7bbc386764bd4_SeqId_47_ 2023-02-13 00:53:58,315 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__o4q7tc7asepj5u8rl71cc9dsch6mfcr7athn20oc15ul5r72qrfjql8e640964ud/family1/80938da918ea47aab571b657b18c1ab7 into 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/75413152bd504844b9f7bbc386764bd4_SeqId_47_ - updating store file list. 2023-02-13 00:53:58,322 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/75413152bd504844b9f7bbc386764bd4_SeqId_47_ into 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1 2023-02-13 00:53:58,322 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__o4q7tc7asepj5u8rl71cc9dsch6mfcr7athn20oc15ul5r72qrfjql8e640964ud/family1/80938da918ea47aab571b657b18c1ab7 into 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/75413152bd504844b9f7bbc386764bd4_SeqId_47_) 2023-02-13 00:53:58,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__o4q7tc7asepj5u8rl71cc9dsch6mfcr7athn20oc15ul5r72qrfjql8e640964ud/family1/80938da918ea47aab571b657b18c1ab7 2023-02-13 00:53:58,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-13 00:53:58,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-13 00:53:58,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(1912): 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1 is initiating minor compaction (all files) 2023-02-13 00:53:58,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@57e970f2; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:53:58,324 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:58,324 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/9749a7005e144eb7997753027f825d16, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/222c3a8ac46046798650e8ea345fa89d, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/75413152bd504844b9f7bbc386764bd4_SeqId_47_] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/.tmp, totalSize=358.6 K 2023-02-13 00:53:58,325 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 9749a7005e144eb7997753027f825d16, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878845884 2023-02-13 00:53:58,325 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 222c3a8ac46046798650e8ea345fa89d, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621245884 2023-02-13 00:53:58,326 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 75413152bd504844b9f7bbc386764bd4_SeqId_47_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878845884 2023-02-13 00:53:58,326 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 00:53:58,327 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:53:58,329 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130ff87fe5f6b804db3854f9702855c2759 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130ff87fe5f6b804db3854f9702855c2759 2023-02-13 00:53:58,332 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120165d7708818b24c55a55bcdbdffe351a5 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120165d7708818b24c55a55bcdbdffe351a5 2023-02-13 00:53:58,334 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205db706bc5b03344d1a1c57e4cb4621409 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205db706bc5b03344d1a1c57e4cb4621409 2023-02-13 00:53:58,334 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-02-13 00:53:58,334 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-13 00:53:58,335 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 00:53:58,335 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 00:53:58,335 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:53:58,336 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-13 00:53:58,338 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 1.6370 sec 2023-02-13 00:53:58,365 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0#family1#compaction#11 average throughput is 6.88 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:53:58,405 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/.tmp/family1/ea0751fb3a9242edafa6f113fc87874f as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/ea0751fb3a9242edafa6f113fc87874f 2023-02-13 00:53:58,414 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1 of 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 into ea0751fb3a9242edafa6f113fc87874f(size=177.2 K), total size for store is 177.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:53:58,414 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0: 2023-02-13 00:53:58,414 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0., storeName=family1, fileCount=3, fileSize=358.6 K (175.8 K, 6.6 K, 176.2 K), priority=2, time=1676249638324; duration=0sec 2023-02-13 00:53:58,414 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:53:58,449 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/9749a7005e144eb7997753027f825d16, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/222c3a8ac46046798650e8ea345fa89d, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/75413152bd504844b9f7bbc386764bd4_SeqId_47_] to archive 2023-02-13 00:53:58,450 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:53:58,453 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/9749a7005e144eb7997753027f825d16 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/9749a7005e144eb7997753027f825d16 2023-02-13 00:53:58,455 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/222c3a8ac46046798650e8ea345fa89d to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/222c3a8ac46046798650e8ea345fa89d 2023-02-13 00:53:58,456 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/75413152bd504844b9f7bbc386764bd4_SeqId_47_ to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/75413152bd504844b9f7bbc386764bd4_SeqId_47_ 2023-02-13 00:53:58,494 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-13 00:53:58,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure2.ProcedureExecutor(1029): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:58,931 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-02-13 00:53:58,932 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-02-13 00:53:58,933 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-02-13 00:53:58,937 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-02-13 00:53:58,951 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, REOPEN/MOVE}] 2023-02-13 00:53:58,953 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, REOPEN/MOVE 2023-02-13 00:53:58,954 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:58,954 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676249638954"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249638954"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249638954"}]},"ts":"1676249638954"} 2023-02-13 00:53:58,957 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:53:59,117 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 2023-02-13 00:53:59,117 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, disabling compactions & flushes 2023-02-13 00:53:59,117 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:59,117 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:59,117 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. after waiting 0 ms 2023-02-13 00:53:59,117 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:59,124 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-13 00:53:59,125 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:59,125 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0: Waiting for close lock at 1676249639117Running coprocessor pre-close hooks at 1676249639117Disabling compacts and flushes for region at 1676249639117Disabling writes for close at 1676249639117Writing region close event to WAL at 1676249639120 (+3 ms)Running coprocessor post-close hooks at 1676249639124 (+4 ms)Closed at 1676249639125 (+1 ms) 2023-02-13 00:53:59,125 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 to self. 2023-02-13 00:53:59,128 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 2023-02-13 00:53:59,128 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, regionState=CLOSED 2023-02-13 00:53:59,128 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676249639128"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249639128"}]},"ts":"1676249639128"} 2023-02-13 00:53:59,132 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-02-13 00:53:59,132 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, server=jenkins-hbase4.apache.org,36605,1676249622318 in 173 msec 2023-02-13 00:53:59,133 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=true 2023-02-13 00:53:59,284 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:59,284 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676249639284"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249639284"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249639284"}]},"ts":"1676249639284"} 2023-02-13 00:53:59,287 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:53:59,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:59,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:53:59,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 2023-02-13 00:53:59,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:53:59,444 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 2023-02-13 00:53:59,444 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 2023-02-13 00:53:59,445 INFO [StoreOpener-9dbd9ef1f4f3ecd6dc6fca04bd7b98f0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 2023-02-13 00:53:59,446 DEBUG [StoreOpener-9dbd9ef1f4f3ecd6dc6fca04bd7b98f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1 2023-02-13 00:53:59,446 DEBUG [StoreOpener-9dbd9ef1f4f3ecd6dc6fca04bd7b98f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1 2023-02-13 00:53:59,447 INFO [StoreOpener-9dbd9ef1f4f3ecd6dc6fca04bd7b98f0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 columnFamilyName family1 2023-02-13 00:53:59,455 DEBUG [StoreOpener-9dbd9ef1f4f3ecd6dc6fca04bd7b98f0-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/ea0751fb3a9242edafa6f113fc87874f 2023-02-13 00:53:59,455 INFO [StoreOpener-9dbd9ef1f4f3ecd6dc6fca04bd7b98f0-1] regionserver.HStore(310): Store=9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:53:59,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 2023-02-13 00:53:59,458 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 2023-02-13 00:53:59,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 2023-02-13 00:53:59,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73898337, jitterRate=0.10117103159427643}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 00:53:59,462 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0: Running coprocessor pre-open hook at 1676249639444Writing region info on filesystem at 1676249639444Initializing all the Stores at 1676249639445 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1676249639445Cleaning up temporary data from old regions at 1676249639458 (+13 ms)Cleaning up detritus from prior splits at 1676249639459 (+1 ms)Running coprocessor post-open hooks at 1676249639462 (+3 ms)Region opened successfully at 1676249639462 2023-02-13 00:53:59,463 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0., pid=17, masterSystemTime=1676249639439 2023-02-13 00:53:59,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:59,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:53:59,466 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:53:59,466 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676249639466"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249639466"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249639466"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249639466"}]},"ts":"1676249639466"} 2023-02-13 00:53:59,470 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-02-13 00:53:59,470 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, server=jenkins-hbase4.apache.org,36605,1676249622318 in 182 msec 2023-02-13 00:53:59,473 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=14 2023-02-13 00:53:59,473 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, REOPEN/MOVE in 519 msec 2023-02-13 00:53:59,477 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-13 00:53:59,477 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 538 msec 2023-02-13 00:53:59,479 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 980 msec 2023-02-13 00:53:59,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=13 2023-02-13 00:53:59,489 INFO [Listener at localhost/37869] compactions.TestMobCompactor(278): alter status finished 2023-02-13 00:53:59,490 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43563] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 00:53:59,492 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-13 00:53:59,493 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-13 00:53:59,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:53:59,494 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676250239494 2023-02-13 00:53:59,495 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 00:53:59,496 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-02-13 00:53:59,496 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 00:53:59,497 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] zookeeper.ReadOnlyZKClient(139): Connect 0x5273cd1c to 127.0.0.1:54478 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:53:59,504 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5a07ea27, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:53:59,504 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 00:53:59,504 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 00:53:59,504 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 00:53:59,504 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 00:53:59,504 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 00:54:00,341 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511280a2e62ed6cb54c53ac4b6825f537976d to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511280a2e62ed6cb54c53ac4b6825f537976d 2023-02-13 00:54:00,342 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 00:54:00,355 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:54:00,356 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41816, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:54:00,364 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 00:54:00,366 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42078, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 00:54:00,378 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/655508965bb3415196332d55f9501047 first=Optional[r05] last=Optional[r8] 2023-02-13 00:54:00,378 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=51 for row with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/655508965bb3415196332d55f9501047}] 2023-02-13 00:54:00,383 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/655508965bb3415196332d55f9501047 for inclusion in 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1 2023-02-13 00:54:00,387 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-13 00:54:00,387 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 00:54:00,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0: 2023-02-13 00:54:00,389 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/655508965bb3415196332d55f9501047 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__fo937lvjcpqolp830a6grj8jervb70g6ce21kfgbdrnrujs571ah3rgs2u43frc9/family1/655508965bb3415196332d55f9501047 2023-02-13 00:54:00,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__fo937lvjcpqolp830a6grj8jervb70g6ce21kfgbdrnrujs571ah3rgs2u43frc9/family1/655508965bb3415196332d55f9501047 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/593b3ed7921e478dad0ea43f2846fa4d_SeqId_53_ 2023-02-13 00:54:00,392 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__fo937lvjcpqolp830a6grj8jervb70g6ce21kfgbdrnrujs571ah3rgs2u43frc9/family1/655508965bb3415196332d55f9501047 into 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/593b3ed7921e478dad0ea43f2846fa4d_SeqId_53_ - updating store file list. 2023-02-13 00:54:00,397 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/593b3ed7921e478dad0ea43f2846fa4d_SeqId_53_ into 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1 2023-02-13 00:54:00,397 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__fo937lvjcpqolp830a6grj8jervb70g6ce21kfgbdrnrujs571ah3rgs2u43frc9/family1/655508965bb3415196332d55f9501047 into 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/family1/593b3ed7921e478dad0ea43f2846fa4d_SeqId_53_) 2023-02-13 00:54:00,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__fo937lvjcpqolp830a6grj8jervb70g6ce21kfgbdrnrujs571ah3rgs2u43frc9/family1/655508965bb3415196332d55f9501047 2023-02-13 00:54:00,400 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 00:54:00,401 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:00,403 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120ab6c17e2149e4850957d8a61f86bebda to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120ab6c17e2149e4850957d8a61f86bebda 2023-02-13 00:54:00,404 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128304da1ac688644c2ad5b4afd85c8813a to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128304da1ac688644c2ad5b4afd85c8813a 2023-02-13 00:54:00,404 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-02-13 00:54:00,404 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-13 00:54:00,405 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 00:54:00,405 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 00:54:00,405 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 00:54:00,405 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 00:54:00,405 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:54:00,406 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-13 00:54:00,408 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 915 msec 2023-02-13 00:54:00,480 INFO [Listener at localhost/37869] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=248 (was 211) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:36605-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:37652 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4576273d-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:37544 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:37464 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:37520 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:37570 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:37590 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:37664 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54478@0x10c3ee4b-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: hconnection-0x4576273d-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54478@0x10c3ee4b sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/151675935.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xc9c8ed0-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:37440 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54478@0x10c3ee4b-SendThread(127.0.0.1:54478) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54478@0x5273cd1c-SendThread(127.0.0.1:54478) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:37458 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xc9c8ed0-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:37482 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1055518213_15 at /127.0.0.1:37468 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:37560 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:37386 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4576273d-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54478@0x5273cd1c-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: (jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:37528 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54478@0x5273cd1c sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/151675935.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:37410 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:37604 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=593 (was 475) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=295 (was 284) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3862 (was 4243) 2023-02-13 00:54:00,495 INFO [Listener at localhost/37869] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=248, OpenFileDescriptor=593, MaxFileDescriptor=60000, SystemLoadAverage=295, ProcessCount=168, AvailableMemoryMB=3860 2023-02-13 00:54:00,497 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-13 00:54:00,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:00,500 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 00:54:00,500 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-02-13 00:54:00,501 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 00:54:00,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 00:54:00,502 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87 2023-02-13 00:54:00,503 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87 empty. 2023-02-13 00:54:00,503 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87 2023-02-13 00:54:00,503 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-02-13 00:54:00,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 00:54:00,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 00:54:00,914 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-13 00:54:00,916 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 7e22638e09b3821acf9ca191933ddf87, NAME => 'testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp 2023-02-13 00:54:01,104 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 00:54:01,322 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:01,323 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1603): Closing 7e22638e09b3821acf9ca191933ddf87, disabling compactions & flushes 2023-02-13 00:54:01,323 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:01,323 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:01,323 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. after waiting 0 ms 2023-02-13 00:54:01,323 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:01,323 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:01,323 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 7e22638e09b3821acf9ca191933ddf87: Waiting for close lock at 1676249641323Disabling compacts and flushes for region at 1676249641323Disabling writes for close at 1676249641323Writing region close event to WAL at 1676249641323Closed at 1676249641323 2023-02-13 00:54:01,326 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 00:54:01,327 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676249641327"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249641327"}]},"ts":"1676249641327"} 2023-02-13 00:54:01,329 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 00:54:01,330 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 00:54:01,330 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249641330"}]},"ts":"1676249641330"} 2023-02-13 00:54:01,332 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-02-13 00:54:01,334 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=7e22638e09b3821acf9ca191933ddf87, ASSIGN}] 2023-02-13 00:54:01,336 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=7e22638e09b3821acf9ca191933ddf87, ASSIGN 2023-02-13 00:54:01,337 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=7e22638e09b3821acf9ca191933ddf87, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=false 2023-02-13 00:54:01,488 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=7e22638e09b3821acf9ca191933ddf87, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:01,488 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676249641488"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249641488"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249641488"}]},"ts":"1676249641488"} 2023-02-13 00:54:01,491 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 7e22638e09b3821acf9ca191933ddf87, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:54:01,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 00:54:01,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:01,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7e22638e09b3821acf9ca191933ddf87, NAME => 'testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:54:01,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy 7e22638e09b3821acf9ca191933ddf87 2023-02-13 00:54:01,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:01,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 7e22638e09b3821acf9ca191933ddf87 2023-02-13 00:54:01,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 7e22638e09b3821acf9ca191933ddf87 2023-02-13 00:54:01,660 INFO [StoreOpener-7e22638e09b3821acf9ca191933ddf87-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 7e22638e09b3821acf9ca191933ddf87 2023-02-13 00:54:01,662 DEBUG [StoreOpener-7e22638e09b3821acf9ca191933ddf87-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1 2023-02-13 00:54:01,662 DEBUG [StoreOpener-7e22638e09b3821acf9ca191933ddf87-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1 2023-02-13 00:54:01,663 INFO [StoreOpener-7e22638e09b3821acf9ca191933ddf87-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7e22638e09b3821acf9ca191933ddf87 columnFamilyName family1 2023-02-13 00:54:01,665 INFO [StoreOpener-7e22638e09b3821acf9ca191933ddf87-1] regionserver.HStore(310): Store=7e22638e09b3821acf9ca191933ddf87/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:01,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87 2023-02-13 00:54:01,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87 2023-02-13 00:54:01,670 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 7e22638e09b3821acf9ca191933ddf87 2023-02-13 00:54:01,672 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:54:01,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 7e22638e09b3821acf9ca191933ddf87; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71077936, jitterRate=0.059143781661987305}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 00:54:01,673 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 7e22638e09b3821acf9ca191933ddf87: Running coprocessor pre-open hook at 1676249641649Writing region info on filesystem at 1676249641649Initializing all the Stores at 1676249641650 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1676249641651 (+1 ms)Cleaning up temporary data from old regions at 1676249641668 (+17 ms)Cleaning up detritus from prior splits at 1676249641668Running coprocessor post-open hooks at 1676249641673 (+5 ms)Region opened successfully at 1676249641673 2023-02-13 00:54:01,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87., pid=21, masterSystemTime=1676249641643 2023-02-13 00:54:01,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:01,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:01,676 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=7e22638e09b3821acf9ca191933ddf87, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:01,677 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676249641676"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249641676"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249641676"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249641676"}]},"ts":"1676249641676"} 2023-02-13 00:54:01,680 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-02-13 00:54:01,680 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 7e22638e09b3821acf9ca191933ddf87, server=jenkins-hbase4.apache.org,36605,1676249622318 in 187 msec 2023-02-13 00:54:01,682 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-02-13 00:54:01,682 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=7e22638e09b3821acf9ca191933ddf87, ASSIGN in 346 msec 2023-02-13 00:54:01,683 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 00:54:01,683 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249641683"}]},"ts":"1676249641683"} 2023-02-13 00:54:01,685 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-02-13 00:54:01,689 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 00:54:01,690 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 1.1920 sec 2023-02-13 00:54:02,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 00:54:02,607 INFO [Listener at localhost/37869] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-02-13 00:54:02,642 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:02,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:02,644 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 00:54:02,644 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:02,645 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 00:54:02,645 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:02,645 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:02,645 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:02,646 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:02,647 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:02,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:02,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:02,647 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:02,647 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:02,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:02,647 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:02,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:02,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:02,648 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:02,648 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:02,648 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 00:54:02,648 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:02,649 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 00:54:02,649 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:02,649 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:02,649 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:02,649 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. started... 2023-02-13 00:54:02,650 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e22638e09b3821acf9ca191933ddf87 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-13 00:54:03,081 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130861b85d33d494050a91e7a7af1f29e3b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130861b85d33d494050a91e7a7af1f29e3b 2023-02-13 00:54:03,487 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/.tmp/family1/69c7e4acc3344889a10b410ac1e77790 2023-02-13 00:54:03,495 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/.tmp/family1/69c7e4acc3344889a10b410ac1e77790 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/69c7e4acc3344889a10b410ac1e77790 2023-02-13 00:54:03,501 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/69c7e4acc3344889a10b410ac1e77790, entries=1000, sequenceid=5, filesize=166.0 K 2023-02-13 00:54:03,502 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 7e22638e09b3821acf9ca191933ddf87 in 852ms, sequenceid=5, compaction requested=false 2023-02-13 00:54:03,502 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 00:54:03,503 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e22638e09b3821acf9ca191933ddf87: 2023-02-13 00:54:03,503 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:03,503 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:03,503 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:03,503 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:03,503 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 00:54:03,504 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 00:54:03,505 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,505 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:03,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:03,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:03,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:03,506 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,506 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:03,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:03,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:03,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:03,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:03,507 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 00:54:03,508 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@80c0af7[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:03,508 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:03,508 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,509 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,509 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:03,509 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 00:54:03,509 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:03,509 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:03,510 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:03,511 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:03,511 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 00:54:03,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:03,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:03,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:03,511 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:03,511 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 00:54:03,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:03,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:03,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:03,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:03,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:03,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:03,515 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:03,515 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 00:54:03,515 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:03,515 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:03,515 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,515 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:03,518 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,518 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:03,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:03,518 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:03,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,519 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:03,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:03,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:03,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:03,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:03,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,520 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:03,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:03,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:03,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:03,523 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:03,523 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,523 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:03,524 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,524 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 00:54:03,524 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:03,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:03,524 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:03,524 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:03,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:03,525 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:03,525 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:03,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:03,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:03,524 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:03,525 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 00:54:03,525 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,525 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,525 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,625 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:03,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:03,632 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:03,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:03,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,633 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 00:54:03,634 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:03,634 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 00:54:03,634 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:03,635 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,635 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,636 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:03,636 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:03,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:03,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:03,636 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:03,636 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:03,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,637 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:03,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,637 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,638 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 00:54:03,638 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:03,638 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 00:54:03,638 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:03,638 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 00:54:03,638 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:03,638 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:03,638 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:03,639 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. started... 2023-02-13 00:54:03,639 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e22638e09b3821acf9ca191933ddf87 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-13 00:54:04,054 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128bf336725865d49f094f330c8a35a7935 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128bf336725865d49f094f330c8a35a7935 2023-02-13 00:54:04,460 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/.tmp/family1/cb8170603c074203b1c9d4fc88f463c0 2023-02-13 00:54:04,467 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/.tmp/family1/cb8170603c074203b1c9d4fc88f463c0 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/cb8170603c074203b1c9d4fc88f463c0 2023-02-13 00:54:04,472 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/cb8170603c074203b1c9d4fc88f463c0, entries=1, sequenceid=9, filesize=5.0 K 2023-02-13 00:54:04,473 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 7e22638e09b3821acf9ca191933ddf87 in 834ms, sequenceid=9, compaction requested=false 2023-02-13 00:54:04,473 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e22638e09b3821acf9ca191933ddf87: 2023-02-13 00:54:04,473 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:04,473 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:04,473 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:04,473 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:04,474 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 00:54:04,474 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 00:54:04,475 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:04,475 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:04,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:04,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:04,475 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,475 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:04,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:04,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:04,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:04,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:04,477 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 00:54:04,477 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7330139c[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:04,477 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:04,477 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,479 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,479 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:04,479 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 00:54:04,479 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:04,479 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:04,479 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:04,481 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:04,481 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 00:54:04,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:04,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:04,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:04,481 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:04,481 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 00:54:04,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:04,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:04,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:04,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:04,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:04,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:04,484 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:04,484 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 00:54:04,484 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:04,484 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:04,484 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,484 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:04,485 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,485 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:04,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:04,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,486 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:04,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:04,486 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:04,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:04,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:04,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:04,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:04,487 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:04,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:04,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:04,493 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:04,493 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:04,493 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:04,493 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:04,493 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:04,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:04,493 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 00:54:04,493 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:04,493 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,494 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:04,494 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,494 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,494 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,494 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:04,494 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:04,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:04,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:04,594 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:04,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:04,600 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:04,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:04,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,603 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 00:54:04,603 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:04,603 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 00:54:04,603 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:04,604 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,604 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,605 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:04,605 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:04,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:04,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:04,605 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:04,605 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:04,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,606 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:04,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,606 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,607 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 00:54:04,607 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:04,607 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 00:54:04,607 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:04,607 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 00:54:04,607 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:04,607 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:04,608 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:04,608 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. started... 2023-02-13 00:54:04,608 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e22638e09b3821acf9ca191933ddf87 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:04,652 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 00:54:05,024 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120106a590490eb94e70b6f17e64482c70cb to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120106a590490eb94e70b6f17e64482c70cb 2023-02-13 00:54:05,430 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/.tmp/family1/fd0bb0eb9b204894b3da672619d84568 2023-02-13 00:54:05,437 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/.tmp/family1/fd0bb0eb9b204894b3da672619d84568 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/fd0bb0eb9b204894b3da672619d84568 2023-02-13 00:54:05,442 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/fd0bb0eb9b204894b3da672619d84568, entries=1, sequenceid=13, filesize=4.9 K 2023-02-13 00:54:05,443 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7e22638e09b3821acf9ca191933ddf87 in 835ms, sequenceid=13, compaction requested=true 2023-02-13 00:54:05,443 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e22638e09b3821acf9ca191933ddf87: 2023-02-13 00:54:05,444 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:05,444 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:05,444 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:05,444 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:05,444 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 00:54:05,444 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 00:54:05,447 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:05,447 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:05,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:05,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:05,447 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,447 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:05,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:05,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:05,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:05,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:05,449 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 00:54:05,449 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@65992a13[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:05,449 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:05,449 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,451 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,451 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:05,451 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:05,451 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:05,451 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 00:54:05,451 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:05,453 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:05,453 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 00:54:05,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:05,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:05,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:05,453 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:05,453 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 00:54:05,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:05,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:05,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:05,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:05,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:05,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:05,456 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:05,456 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 00:54:05,456 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:05,456 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:05,456 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,456 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:05,458 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,458 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,458 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:05,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:05,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:05,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:05,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:05,459 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:05,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:05,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:05,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,460 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:05,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:05,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:05,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:05,464 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:05,464 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:05,464 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:05,464 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 00:54:05,464 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:05,464 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:05,464 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,464 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:05,465 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:05,464 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 00:54:05,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:05,465 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:05,465 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:05,465 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,466 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,466 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:05,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:05,566 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:05,567 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:05,571 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:05,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:05,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,573 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 00:54:05,573 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:05,574 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 00:54:05,574 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:05,574 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,574 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,576 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:05,576 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:05,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:05,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:05,576 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:05,576 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:05,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,577 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:05,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,577 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,577 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 00:54:05,577 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:05,577 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 00:54:05,577 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:05,578 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 00:54:05,578 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:05,578 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:05,578 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:05,578 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. started... 2023-02-13 00:54:05,578 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e22638e09b3821acf9ca191933ddf87 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:05,994 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205599efef822a64878bf54765d511fa8a1 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205599efef822a64878bf54765d511fa8a1 2023-02-13 00:54:06,400 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/.tmp/family1/edd2247998dd491690a025bcebf0acee 2023-02-13 00:54:06,407 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/.tmp/family1/edd2247998dd491690a025bcebf0acee as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/edd2247998dd491690a025bcebf0acee 2023-02-13 00:54:06,413 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/edd2247998dd491690a025bcebf0acee, entries=1, sequenceid=17, filesize=4.9 K 2023-02-13 00:54:06,414 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7e22638e09b3821acf9ca191933ddf87 in 836ms, sequenceid=17, compaction requested=true 2023-02-13 00:54:06,414 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e22638e09b3821acf9ca191933ddf87: 2023-02-13 00:54:06,414 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:06,414 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:06,414 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:06,414 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:06,414 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 00:54:06,415 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 00:54:06,416 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:06,416 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:06,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:06,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:06,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:06,417 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,417 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:06,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:06,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:06,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:06,418 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 00:54:06,418 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@68b9e83e[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:06,418 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:06,418 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,419 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,419 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:06,420 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 00:54:06,420 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:06,420 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:06,420 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:06,421 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:06,421 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 00:54:06,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:06,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:06,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:06,421 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:06,421 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 00:54:06,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:06,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:06,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:06,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:06,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:06,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:06,424 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:06,424 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 00:54:06,425 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:06,425 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:06,425 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,425 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:06,428 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,428 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:06,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:06,428 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:06,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:06,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:06,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:06,428 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:06,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:06,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:06,430 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:06,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:06,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:06,432 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:06,432 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:06,432 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:06,432 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:06,432 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 00:54:06,432 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:06,432 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,432 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:06,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:06,433 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:06,433 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:06,433 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,433 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:06,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:06,433 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:06,434 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,534 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:06,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:06,540 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:06,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:06,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,543 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 00:54:06,543 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:06,543 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 00:54:06,544 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:06,544 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,544 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,546 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:06,546 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:06,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:06,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:06,546 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:06,546 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:06,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,547 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:06,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,547 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,551 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 00:54:06,551 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:06,551 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 00:54:06,551 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:06,551 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 00:54:06,553 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:06,553 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:06,553 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:06,553 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. started... 2023-02-13 00:54:06,553 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e22638e09b3821acf9ca191933ddf87 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:06,973 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228e13c02e7eba749e9a6194f2264c466c0 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228e13c02e7eba749e9a6194f2264c466c0 2023-02-13 00:54:07,379 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/.tmp/family1/ef5675adcb0342f0a5cf16650af26366 2023-02-13 00:54:07,386 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/.tmp/family1/ef5675adcb0342f0a5cf16650af26366 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/ef5675adcb0342f0a5cf16650af26366 2023-02-13 00:54:07,392 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/ef5675adcb0342f0a5cf16650af26366, entries=1, sequenceid=21, filesize=4.9 K 2023-02-13 00:54:07,393 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7e22638e09b3821acf9ca191933ddf87 in 840ms, sequenceid=21, compaction requested=true 2023-02-13 00:54:07,393 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e22638e09b3821acf9ca191933ddf87: 2023-02-13 00:54:07,393 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:07,394 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:07,394 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:07,394 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:07,394 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 00:54:07,394 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 00:54:07,396 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:07,396 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:07,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:07,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:07,396 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,396 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:07,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:07,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:07,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:07,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:07,398 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 00:54:07,398 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@8a0c6c[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:07,398 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:07,398 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,401 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,401 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:07,401 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 00:54:07,401 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:07,401 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:07,401 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:07,403 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 00:54:07,403 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:07,403 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:07,403 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 00:54:07,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:07,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:07,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:07,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:07,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:07,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:07,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:07,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:07,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:07,406 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:07,406 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 00:54:07,406 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:07,406 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:07,406 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,406 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:07,411 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,412 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:07,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:07,412 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:07,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:07,412 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:07,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:07,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:07,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:07,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,414 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:07,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:07,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:07,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:07,416 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:07,416 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:07,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:07,416 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 00:54:07,416 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:07,416 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 00:54:07,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:07,416 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:07,417 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:07,417 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:07,416 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:07,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:07,417 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:07,417 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,418 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:07,418 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,418 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,418 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,517 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:07,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:07,523 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:07,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:07,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,525 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 00:54:07,525 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:07,526 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 00:54:07,526 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:07,527 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,527 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,528 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:07,528 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:07,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:07,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:07,529 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:07,529 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:07,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,529 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:07,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,530 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,533 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 00:54:07,533 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:07,533 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 00:54:07,533 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:07,534 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 00:54:07,534 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:07,534 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:07,534 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:07,534 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. started... 2023-02-13 00:54:07,534 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e22638e09b3821acf9ca191933ddf87 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:07,962 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231668e3d1e930b40f88a566e0b9772a40c to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231668e3d1e930b40f88a566e0b9772a40c 2023-02-13 00:54:08,373 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/.tmp/family1/f3eb364167fd4a45840635fbf05428f2 2023-02-13 00:54:08,381 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/.tmp/family1/f3eb364167fd4a45840635fbf05428f2 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/f3eb364167fd4a45840635fbf05428f2 2023-02-13 00:54:08,387 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/f3eb364167fd4a45840635fbf05428f2, entries=1, sequenceid=25, filesize=4.9 K 2023-02-13 00:54:08,388 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7e22638e09b3821acf9ca191933ddf87 in 854ms, sequenceid=25, compaction requested=true 2023-02-13 00:54:08,388 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e22638e09b3821acf9ca191933ddf87: 2023-02-13 00:54:08,388 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:08,388 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:08,388 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:08,388 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,388 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 00:54:08,388 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 00:54:08,394 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,394 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:08,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:08,394 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,395 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:08,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:08,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:08,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:08,396 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 00:54:08,396 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@73d55312[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:08,396 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:08,396 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,397 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,398 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:08,398 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 00:54:08,398 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:08,398 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,398 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:08,399 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 00:54:08,399 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:08,399 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,399 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 00:54:08,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:08,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:08,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:08,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:08,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:08,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:08,403 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:08,403 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 00:54:08,403 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:08,403 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:08,403 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,403 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:08,404 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,404 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,404 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:08,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:08,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:08,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:08,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:08,405 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:08,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:08,406 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:08,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,410 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,410 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,410 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,410 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:08,410 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:08,410 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:08,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:08,411 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,411 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:08,411 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 00:54:08,412 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:08,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:08,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:08,412 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,412 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,412 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 00:54:08,412 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:08,413 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:08,513 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:08,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:08,519 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:08,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:08,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,524 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 00:54:08,524 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:08,524 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 00:54:08,524 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:08,525 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,525 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,526 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:08,526 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:08,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:08,527 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,527 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:08,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,527 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:08,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,528 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,528 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 00:54:08,528 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,528 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 00:54:08,528 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:08,529 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 00:54:08,529 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:08,529 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:08,529 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:08,529 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. started... 2023-02-13 00:54:08,529 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e22638e09b3821acf9ca191933ddf87 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:08,552 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601019a1e6ebf7e9946579bd7cbf909ad205b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601019a1e6ebf7e9946579bd7cbf909ad205b 2023-02-13 00:54:08,965 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/.tmp/family1/6e9c60506f7a4034a642ca3674adad8d 2023-02-13 00:54:08,973 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/.tmp/family1/6e9c60506f7a4034a642ca3674adad8d as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/6e9c60506f7a4034a642ca3674adad8d 2023-02-13 00:54:08,978 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/6e9c60506f7a4034a642ca3674adad8d, entries=1, sequenceid=29, filesize=4.9 K 2023-02-13 00:54:08,979 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7e22638e09b3821acf9ca191933ddf87 in 450ms, sequenceid=29, compaction requested=true 2023-02-13 00:54:08,979 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e22638e09b3821acf9ca191933ddf87: 2023-02-13 00:54:08,979 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:08,980 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:08,980 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:08,980 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,980 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 00:54:08,980 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 00:54:08,981 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,981 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:08,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:08,982 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,982 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:08,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:08,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:08,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:08,984 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 00:54:08,984 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5f3036c9[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:08,984 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:08,984 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,985 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,985 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:08,985 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 00:54:08,985 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:08,985 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,985 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:08,991 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,991 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 00:54:08,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:08,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:08,991 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:08,991 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 00:54:08,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:08,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:08,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:08,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:08,994 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:08,994 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 00:54:08,994 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:08,994 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:08,994 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,994 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:08,996 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,996 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:08,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:08,996 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:08,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:08,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:08,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:08,997 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:08,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,998 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:08,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:08,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:08,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:09,000 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:09,000 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:09,000 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,000 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:09,000 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,000 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 00:54:09,001 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:09,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:09,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:09,000 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:09,001 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:09,001 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,001 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,001 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:09,001 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,002 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:09,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:09,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:09,102 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:09,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:09,110 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:09,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:09,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,112 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 00:54:09,112 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:09,113 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 00:54:09,113 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:09,113 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,113 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,114 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:09,114 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:09,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:09,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:09,114 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:09,115 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:09,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,115 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:09,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,115 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,116 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 00:54:09,116 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,116 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 00:54:09,116 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:09,116 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 00:54:09,116 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:09,117 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:09,117 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:09,117 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. started... 2023-02-13 00:54:09,117 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e22638e09b3821acf9ca191933ddf87 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:09,536 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601030f7a3f1842c74a3ba20bdd0319f7183b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601030f7a3f1842c74a3ba20bdd0319f7183b 2023-02-13 00:54:09,941 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/.tmp/family1/e1d9b2ac65ef4d2085107c7df83bdaec 2023-02-13 00:54:09,948 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/.tmp/family1/e1d9b2ac65ef4d2085107c7df83bdaec as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/e1d9b2ac65ef4d2085107c7df83bdaec 2023-02-13 00:54:09,953 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/e1d9b2ac65ef4d2085107c7df83bdaec, entries=1, sequenceid=33, filesize=4.9 K 2023-02-13 00:54:09,954 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7e22638e09b3821acf9ca191933ddf87 in 837ms, sequenceid=33, compaction requested=true 2023-02-13 00:54:09,954 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e22638e09b3821acf9ca191933ddf87: 2023-02-13 00:54:09,954 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:09,954 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:09,954 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:09,954 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:09,955 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 00:54:09,955 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 00:54:09,957 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:09,957 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:09,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:09,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:09,957 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,957 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:09,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:09,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:09,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:09,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:09,959 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 00:54:09,959 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@786fcd09[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:09,959 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:09,959 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,960 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,960 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:09,960 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 00:54:09,960 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:09,960 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:09,960 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:09,965 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 00:54:09,965 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:09,965 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:09,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:09,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:09,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:09,965 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 00:54:09,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:09,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:09,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:09,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:09,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:09,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:09,968 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:09,968 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 00:54:09,968 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:09,968 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:09,968 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,968 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:09,970 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,970 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:09,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:09,970 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:09,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:09,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:09,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:09,971 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:09,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:09,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:09,972 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:09,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:09,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:09,974 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:09,974 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,974 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:09,974 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:09,974 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 00:54:09,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:09,974 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:09,974 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 00:54:09,974 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:09,974 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:09,974 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:09,974 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:09,975 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:09,975 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,975 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:09,975 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:09,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:10,075 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:10,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:10,081 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:10,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:10,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,084 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 00:54:10,084 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:10,084 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 00:54:10,084 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:10,085 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,085 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,086 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:10,086 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:10,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:10,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:10,086 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:10,086 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:10,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,087 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:10,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,087 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,087 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 00:54:10,087 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,087 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 00:54:10,087 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:10,088 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 00:54:10,088 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:10,088 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:10,088 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:10,088 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. started... 2023-02-13 00:54:10,088 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e22638e09b3821acf9ca191933ddf87 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:10,102 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090707cd00b9f4e842339fb391280c6ea8b7 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015090707cd00b9f4e842339fb391280c6ea8b7 2023-02-13 00:54:10,508 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/.tmp/family1/d509cad196b64eb49a8e0199d87095ad 2023-02-13 00:54:10,515 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/.tmp/family1/d509cad196b64eb49a8e0199d87095ad as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/d509cad196b64eb49a8e0199d87095ad 2023-02-13 00:54:10,520 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/d509cad196b64eb49a8e0199d87095ad, entries=1, sequenceid=37, filesize=4.9 K 2023-02-13 00:54:10,521 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7e22638e09b3821acf9ca191933ddf87 in 433ms, sequenceid=37, compaction requested=true 2023-02-13 00:54:10,521 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e22638e09b3821acf9ca191933ddf87: 2023-02-13 00:54:10,521 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:10,521 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:10,521 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:10,521 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:10,521 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 00:54:10,521 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 00:54:10,528 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:10,528 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:10,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:10,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:10,529 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,529 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:10,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:10,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:10,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:10,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:10,530 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 00:54:10,530 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@28d64704[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:10,530 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:10,530 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,531 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,531 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:10,531 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 00:54:10,531 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:10,531 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:10,531 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:10,533 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 00:54:10,533 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:10,533 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:10,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:10,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:10,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:10,533 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 00:54:10,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:10,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:10,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:10,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:10,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:10,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:10,535 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:10,535 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 00:54:10,535 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:10,535 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:10,535 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,535 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:10,537 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,537 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:10,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:10,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,537 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:10,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:10,537 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:10,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:10,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:10,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:10,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,538 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:10,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:10,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:10,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:10,541 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:10,541 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,541 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:10,541 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:10,541 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:10,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:10,541 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:10,541 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 00:54:10,542 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:10,541 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,542 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 00:54:10,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:10,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:10,542 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:10,542 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,542 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:10,542 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,543 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:10,543 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,643 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:10,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:10,648 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:10,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:10,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,650 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 00:54:10,650 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:10,651 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 00:54:10,651 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:10,651 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,651 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,652 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:10,652 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:10,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:10,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:10,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,653 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:10,653 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:10,653 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:10,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,653 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,654 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 00:54:10,654 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:10,654 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 00:54:10,654 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:10,655 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 00:54:10,655 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:10,655 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:10,655 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:10,656 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. started... 2023-02-13 00:54:10,656 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e22638e09b3821acf9ca191933ddf87 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:11,071 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112092c3b0a1d00047d99988505f1ba87975 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112092c3b0a1d00047d99988505f1ba87975 2023-02-13 00:54:11,477 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/.tmp/family1/441d6d130f094949bd4cb33f1f83f9be 2023-02-13 00:54:11,483 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/.tmp/family1/441d6d130f094949bd4cb33f1f83f9be as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/441d6d130f094949bd4cb33f1f83f9be 2023-02-13 00:54:11,488 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/441d6d130f094949bd4cb33f1f83f9be, entries=1, sequenceid=41, filesize=4.9 K 2023-02-13 00:54:11,489 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7e22638e09b3821acf9ca191933ddf87 in 833ms, sequenceid=41, compaction requested=true 2023-02-13 00:54:11,489 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e22638e09b3821acf9ca191933ddf87: 2023-02-13 00:54:11,489 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:11,489 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:11,489 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:11,489 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:11,489 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 00:54:11,489 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 00:54:11,491 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:11,491 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:11,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:11,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:11,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:11,491 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:11,491 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:11,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:11,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:11,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:11,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:11,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:11,492 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 00:54:11,492 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@71c2cccf[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:11,492 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:11,492 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:11,497 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:11,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:11,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:11,498 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:11,498 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 00:54:11,498 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:11,498 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:11,498 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:11,499 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:11,499 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 00:54:11,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:11,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:11,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:11,499 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:11,499 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 00:54:11,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:11,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:11,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:11,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:11,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:11,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:11,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:11,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:11,502 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:11,502 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 00:54:11,502 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:11,502 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:11,502 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:11,502 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:11,504 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:11,504 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:11,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:11,504 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:11,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:11,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:11,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:11,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:11,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:11,504 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:11,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:11,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:11,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:11,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:11,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:11,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:11,505 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:11,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:11,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:11,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:11,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:11,507 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:11,507 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:11,508 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:11,508 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:11,508 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:11,508 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 00:54:11,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:11,508 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 00:54:11,509 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:11,508 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:11,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:11,509 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:11,509 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:11,509 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:11,509 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:11,509 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:11,509 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:11,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:11,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:11,610 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:11,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:11,612 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43563] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-02-13 00:54:11,614 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] procedure2.ProcedureExecutor(1029): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 00:54:11,614 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 00:54:11,615 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:54:11,615 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676250251615 2023-02-13 00:54:11,616 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 00:54:11,618 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-02-13 00:54:11,618 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 00:54:11,618 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] zookeeper.ReadOnlyZKClient(139): Connect 0x3a7fa659 to 127.0.0.1:54478 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:54:11,623 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5fcc35e3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:54:11,624 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 00:54:11,652 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103ea584956ed484475ac124956e72419f5 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103ea584956ed484475ac124956e72419f5 2023-02-13 00:54:11,653 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 00:54:11,666 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:54:11,667 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51584, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:54:11,674 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 00:54:11,675 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37854, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 00:54:11,685 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/fe45b766a5ce4011bb3b98313df1ea43 first=Optional[r5] last=Optional[r6] 2023-02-13 00:54:11,685 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/fe45b766a5ce4011bb3b98313df1ea43}] 2023-02-13 00:54:11,689 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/fe45b766a5ce4011bb3b98313df1ea43 for inclusion in 7e22638e09b3821acf9ca191933ddf87/family1 2023-02-13 00:54:11,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-02-13 00:54:11,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 00:54:11,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for 7e22638e09b3821acf9ca191933ddf87: 2023-02-13 00:54:11,693 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/fe45b766a5ce4011bb3b98313df1ea43 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__np7g9cpoik68ckvpngrfu54en6qf3hjuiucjd2dq5j28p6q2m5kkecf6jvme2vs1/family1/fe45b766a5ce4011bb3b98313df1ea43 2023-02-13 00:54:11,696 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__np7g9cpoik68ckvpngrfu54en6qf3hjuiucjd2dq5j28p6q2m5kkecf6jvme2vs1/family1/fe45b766a5ce4011bb3b98313df1ea43 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/99a5414e3b9e4a76ace4d05bbbdae7dd_SeqId_44_ 2023-02-13 00:54:11,696 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__np7g9cpoik68ckvpngrfu54en6qf3hjuiucjd2dq5j28p6q2m5kkecf6jvme2vs1/family1/fe45b766a5ce4011bb3b98313df1ea43 into 7e22638e09b3821acf9ca191933ddf87/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/99a5414e3b9e4a76ace4d05bbbdae7dd_SeqId_44_ - updating store file list. 2023-02-13 00:54:11,701 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/99a5414e3b9e4a76ace4d05bbbdae7dd_SeqId_44_ into 7e22638e09b3821acf9ca191933ddf87/family1 2023-02-13 00:54:11,701 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__np7g9cpoik68ckvpngrfu54en6qf3hjuiucjd2dq5j28p6q2m5kkecf6jvme2vs1/family1/fe45b766a5ce4011bb3b98313df1ea43 into 7e22638e09b3821acf9ca191933ddf87/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/99a5414e3b9e4a76ace4d05bbbdae7dd_SeqId_44_) 2023-02-13 00:54:11,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__np7g9cpoik68ckvpngrfu54en6qf3hjuiucjd2dq5j28p6q2m5kkecf6jvme2vs1/family1/fe45b766a5ce4011bb3b98313df1ea43 2023-02-13 00:54:11,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-13 00:54:11,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-13 00:54:11,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(1912): 7e22638e09b3821acf9ca191933ddf87/family1 is initiating minor compaction 2023-02-13 00:54:11,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4048b01; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:11,705 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 7e22638e09b3821acf9ca191933ddf87/family1 in testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:11,705 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/cb8170603c074203b1c9d4fc88f463c0, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/fd0bb0eb9b204894b3da672619d84568, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/edd2247998dd491690a025bcebf0acee, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/ef5675adcb0342f0a5cf16650af26366, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/f3eb364167fd4a45840635fbf05428f2, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/6e9c60506f7a4034a642ca3674adad8d, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/e1d9b2ac65ef4d2085107c7df83bdaec, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/d509cad196b64eb49a8e0199d87095ad, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/441d6d130f094949bd4cb33f1f83f9be, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/99a5414e3b9e4a76ace4d05bbbdae7dd_SeqId_44_] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/.tmp, totalSize=49.7 K 2023-02-13 00:54:11,706 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting cb8170603c074203b1c9d4fc88f463c0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-13 00:54:11,706 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting fd0bb0eb9b204894b3da672619d84568, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-13 00:54:11,706 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting edd2247998dd491690a025bcebf0acee, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-13 00:54:11,706 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting ef5675adcb0342f0a5cf16650af26366, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-13 00:54:11,706 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting f3eb364167fd4a45840635fbf05428f2, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-13 00:54:11,706 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 6e9c60506f7a4034a642ca3674adad8d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-13 00:54:11,706 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting e1d9b2ac65ef4d2085107c7df83bdaec, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-13 00:54:11,706 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting d509cad196b64eb49a8e0199d87095ad, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-13 00:54:11,706 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 441d6d130f094949bd4cb33f1f83f9be, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-13 00:54:11,706 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 99a5414e3b9e4a76ace4d05bbbdae7dd_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-13 00:54:11,707 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 00:54:11,708 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:11,710 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601019a1e6ebf7e9946579bd7cbf909ad205b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601019a1e6ebf7e9946579bd7cbf909ad205b 2023-02-13 00:54:11,711 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601030f7a3f1842c74a3ba20bdd0319f7183b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601030f7a3f1842c74a3ba20bdd0319f7183b 2023-02-13 00:54:11,712 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-02-13 00:54:11,712 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-02-13 00:54:11,712 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 00:54:11,712 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 00:54:11,755 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7e22638e09b3821acf9ca191933ddf87#family1#compaction#22 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:54:12,175 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/.tmp/family1/5c3cb5a8ac45418cb9c7c2f4fb5402ce as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/5c3cb5a8ac45418cb9c7c2f4fb5402ce 2023-02-13 00:54:12,182 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 7e22638e09b3821acf9ca191933ddf87/family1 of 7e22638e09b3821acf9ca191933ddf87 into 5c3cb5a8ac45418cb9c7c2f4fb5402ce(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:54:12,182 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 7e22638e09b3821acf9ca191933ddf87: 2023-02-13 00:54:12,182 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1676249651705; duration=0sec 2023-02-13 00:54:12,182 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:12,261 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/cb8170603c074203b1c9d4fc88f463c0, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/fd0bb0eb9b204894b3da672619d84568, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/edd2247998dd491690a025bcebf0acee, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/ef5675adcb0342f0a5cf16650af26366, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/f3eb364167fd4a45840635fbf05428f2, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/6e9c60506f7a4034a642ca3674adad8d, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/e1d9b2ac65ef4d2085107c7df83bdaec, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/d509cad196b64eb49a8e0199d87095ad, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/441d6d130f094949bd4cb33f1f83f9be, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/99a5414e3b9e4a76ace4d05bbbdae7dd_SeqId_44_] to archive 2023-02-13 00:54:12,262 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:12,264 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/cb8170603c074203b1c9d4fc88f463c0 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/cb8170603c074203b1c9d4fc88f463c0 2023-02-13 00:54:12,265 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/fd0bb0eb9b204894b3da672619d84568 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/fd0bb0eb9b204894b3da672619d84568 2023-02-13 00:54:12,267 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/edd2247998dd491690a025bcebf0acee to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/edd2247998dd491690a025bcebf0acee 2023-02-13 00:54:12,268 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/ef5675adcb0342f0a5cf16650af26366 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/ef5675adcb0342f0a5cf16650af26366 2023-02-13 00:54:12,269 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/f3eb364167fd4a45840635fbf05428f2 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/f3eb364167fd4a45840635fbf05428f2 2023-02-13 00:54:12,271 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/6e9c60506f7a4034a642ca3674adad8d to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/6e9c60506f7a4034a642ca3674adad8d 2023-02-13 00:54:12,272 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/e1d9b2ac65ef4d2085107c7df83bdaec to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/e1d9b2ac65ef4d2085107c7df83bdaec 2023-02-13 00:54:12,273 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/d509cad196b64eb49a8e0199d87095ad to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/d509cad196b64eb49a8e0199d87095ad 2023-02-13 00:54:12,274 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/441d6d130f094949bd4cb33f1f83f9be to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/441d6d130f094949bd4cb33f1f83f9be 2023-02-13 00:54:12,275 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/99a5414e3b9e4a76ace4d05bbbdae7dd_SeqId_44_ to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/99a5414e3b9e4a76ace4d05bbbdae7dd_SeqId_44_ 2023-02-13 00:54:12,580 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231e7a26328b5f141a387d2f1dcffb26aba to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231e7a26328b5f141a387d2f1dcffb26aba 2023-02-13 00:54:12,581 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 00:54:12,606 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/788589bacb5c45fbbaa8c7805c50657d first=Optional[r1] last=Optional[r4] 2023-02-13 00:54:12,606 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/788589bacb5c45fbbaa8c7805c50657d}] 2023-02-13 00:54:12,608 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/788589bacb5c45fbbaa8c7805c50657d for inclusion in 7e22638e09b3821acf9ca191933ddf87/family1 2023-02-13 00:54:12,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-02-13 00:54:12,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 00:54:12,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for 7e22638e09b3821acf9ca191933ddf87: 2023-02-13 00:54:12,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/788589bacb5c45fbbaa8c7805c50657d to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__e14blp3j3ir4btrmungofbp0mqqgpv46te7eledoi6skm1lmp3hvqfd9t3407drq/family1/788589bacb5c45fbbaa8c7805c50657d 2023-02-13 00:54:12,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__e14blp3j3ir4btrmungofbp0mqqgpv46te7eledoi6skm1lmp3hvqfd9t3407drq/family1/788589bacb5c45fbbaa8c7805c50657d as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/6106cbbff8754871b8f8781ef7c1b0a6_SeqId_47_ 2023-02-13 00:54:12,616 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__e14blp3j3ir4btrmungofbp0mqqgpv46te7eledoi6skm1lmp3hvqfd9t3407drq/family1/788589bacb5c45fbbaa8c7805c50657d into 7e22638e09b3821acf9ca191933ddf87/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/6106cbbff8754871b8f8781ef7c1b0a6_SeqId_47_ - updating store file list. 2023-02-13 00:54:12,621 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/6106cbbff8754871b8f8781ef7c1b0a6_SeqId_47_ into 7e22638e09b3821acf9ca191933ddf87/family1 2023-02-13 00:54:12,621 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__e14blp3j3ir4btrmungofbp0mqqgpv46te7eledoi6skm1lmp3hvqfd9t3407drq/family1/788589bacb5c45fbbaa8c7805c50657d into 7e22638e09b3821acf9ca191933ddf87/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/6106cbbff8754871b8f8781ef7c1b0a6_SeqId_47_) 2023-02-13 00:54:12,621 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__e14blp3j3ir4btrmungofbp0mqqgpv46te7eledoi6skm1lmp3hvqfd9t3407drq/family1/788589bacb5c45fbbaa8c7805c50657d 2023-02-13 00:54:12,621 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-13 00:54:12,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-13 00:54:12,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(1912): 7e22638e09b3821acf9ca191933ddf87/family1 is initiating minor compaction (all files) 2023-02-13 00:54:12,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@25854161; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:12,623 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 7e22638e09b3821acf9ca191933ddf87/family1 in testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:54:12,623 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/69c7e4acc3344889a10b410ac1e77790, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/5c3cb5a8ac45418cb9c7c2f4fb5402ce, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/6106cbbff8754871b8f8781ef7c1b0a6_SeqId_47_] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/.tmp, totalSize=178.1 K 2023-02-13 00:54:12,623 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 69c7e4acc3344889a10b410ac1e77790, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878845884 2023-02-13 00:54:12,624 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 5c3cb5a8ac45418cb9c7c2f4fb5402ce, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621245884 2023-02-13 00:54:12,625 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 6106cbbff8754871b8f8781ef7c1b0a6_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965245884 2023-02-13 00:54:12,625 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 00:54:12,625 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:12,627 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120106a590490eb94e70b6f17e64482c70cb to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120106a590490eb94e70b6f17e64482c70cb 2023-02-13 00:54:12,628 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205599efef822a64878bf54765d511fa8a1 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205599efef822a64878bf54765d511fa8a1 2023-02-13 00:54:12,630 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228e13c02e7eba749e9a6194f2264c466c0 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228e13c02e7eba749e9a6194f2264c466c0 2023-02-13 00:54:12,631 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231668e3d1e930b40f88a566e0b9772a40c to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231668e3d1e930b40f88a566e0b9772a40c 2023-02-13 00:54:12,631 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-02-13 00:54:12,631 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-02-13 00:54:12,632 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 00:54:12,632 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 00:54:12,643 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7e22638e09b3821acf9ca191933ddf87#family1#compaction#23 average throughput is 15.56 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:54:13,466 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/.tmp/family1/3d5b2ab3c23c4aa09f639a0ba22c8f46 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/3d5b2ab3c23c4aa09f639a0ba22c8f46 2023-02-13 00:54:13,472 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 7e22638e09b3821acf9ca191933ddf87/family1 of 7e22638e09b3821acf9ca191933ddf87 into 3d5b2ab3c23c4aa09f639a0ba22c8f46(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:54:13,473 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 7e22638e09b3821acf9ca191933ddf87: 2023-02-13 00:54:13,473 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1676249652623; duration=0sec 2023-02-13 00:54:13,473 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:13,479 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113076b54b5a2a08488bbb9277dd25bcbf3f to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015113076b54b5a2a08488bbb9277dd25bcbf3f 2023-02-13 00:54:13,480 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 00:54:13,505 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b46237d5c08e42589f5bf33e1cf86318 first=Optional[r00] last=Optional[r8] 2023-02-13 00:54:13,506 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b46237d5c08e42589f5bf33e1cf86318}] 2023-02-13 00:54:13,508 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b46237d5c08e42589f5bf33e1cf86318 for inclusion in 7e22638e09b3821acf9ca191933ddf87/family1 2023-02-13 00:54:13,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-02-13 00:54:13,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 00:54:13,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for 7e22638e09b3821acf9ca191933ddf87: 2023-02-13 00:54:13,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b46237d5c08e42589f5bf33e1cf86318 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ckfie9fv7oc2ubjne64fam1fl3grib1n2h32ja7ov60ga7nm9ui8ojtgc1vvl215/family1/b46237d5c08e42589f5bf33e1cf86318 2023-02-13 00:54:13,515 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ckfie9fv7oc2ubjne64fam1fl3grib1n2h32ja7ov60ga7nm9ui8ojtgc1vvl215/family1/b46237d5c08e42589f5bf33e1cf86318 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/31dc75aed3084edab77037e55578ec3f_SeqId_50_ 2023-02-13 00:54:13,516 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ckfie9fv7oc2ubjne64fam1fl3grib1n2h32ja7ov60ga7nm9ui8ojtgc1vvl215/family1/b46237d5c08e42589f5bf33e1cf86318 into 7e22638e09b3821acf9ca191933ddf87/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/31dc75aed3084edab77037e55578ec3f_SeqId_50_ - updating store file list. 2023-02-13 00:54:13,520 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/31dc75aed3084edab77037e55578ec3f_SeqId_50_ into 7e22638e09b3821acf9ca191933ddf87/family1 2023-02-13 00:54:13,520 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ckfie9fv7oc2ubjne64fam1fl3grib1n2h32ja7ov60ga7nm9ui8ojtgc1vvl215/family1/b46237d5c08e42589f5bf33e1cf86318 into 7e22638e09b3821acf9ca191933ddf87/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/31dc75aed3084edab77037e55578ec3f_SeqId_50_) 2023-02-13 00:54:13,521 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ckfie9fv7oc2ubjne64fam1fl3grib1n2h32ja7ov60ga7nm9ui8ojtgc1vvl215/family1/b46237d5c08e42589f5bf33e1cf86318 2023-02-13 00:54:13,523 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 00:54:13,523 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:13,525 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112092c3b0a1d00047d99988505f1ba87975 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112092c3b0a1d00047d99988505f1ba87975 2023-02-13 00:54:13,527 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128bf336725865d49f094f330c8a35a7935 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128bf336725865d49f094f330c8a35a7935 2023-02-13 00:54:13,528 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130861b85d33d494050a91e7a7af1f29e3b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130861b85d33d494050a91e7a7af1f29e3b 2023-02-13 00:54:13,528 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-02-13 00:54:13,528 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-13 00:54:13,529 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 00:54:13,529 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 00:54:13,529 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 00:54:13,529 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 00:54:13,529 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:54:13,530 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 00:54:13,531 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.9170 sec 2023-02-13 00:54:13,547 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/69c7e4acc3344889a10b410ac1e77790, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/5c3cb5a8ac45418cb9c7c2f4fb5402ce, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/6106cbbff8754871b8f8781ef7c1b0a6_SeqId_47_] to archive 2023-02-13 00:54:13,548 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:13,550 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/69c7e4acc3344889a10b410ac1e77790 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/69c7e4acc3344889a10b410ac1e77790 2023-02-13 00:54:13,551 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/5c3cb5a8ac45418cb9c7c2f4fb5402ce to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/5c3cb5a8ac45418cb9c7c2f4fb5402ce 2023-02-13 00:54:13,552 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/6106cbbff8754871b8f8781ef7c1b0a6_SeqId_47_ to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/family1/6106cbbff8754871b8f8781ef7c1b0a6_SeqId_47_ 2023-02-13 00:54:13,626 INFO [Listener at localhost/37869] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=257 (was 248) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54478@0x3a7fa659 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/151675935.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:50256 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1055518213_15 at /127.0.0.1:50290 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2b77bee8-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:50378 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/cluster_0ce39707-0279-6124-1e28-d0d9ced157d1/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:50298 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54478@0x3a7fa659-SendThread(127.0.0.1:54478) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:50308 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1055518213_15 at /127.0.0.1:50318 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x241edf12-shared-pool-23 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2b77bee8-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2b77bee8-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54478@0x3a7fa659-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:50202 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:50416 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:50186 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2b77bee8-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/cluster_0ce39707-0279-6124-1e28-d0d9ced157d1/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2b77bee8-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:50362 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2b77bee8-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:50268 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:50234 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:50400 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2b77bee8-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:50332 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:50384 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:50462 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-21463395_15 at /127.0.0.1:50276 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=615 (was 593) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=236 (was 295), ProcessCount=168 (was 168), AvailableMemoryMB=4440 (was 3860) - AvailableMemoryMB LEAK? - 2023-02-13 00:54:13,638 INFO [Listener at localhost/37869] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=257, OpenFileDescriptor=615, MaxFileDescriptor=60000, SystemLoadAverage=236, ProcessCount=168, AvailableMemoryMB=4439 2023-02-13 00:54:13,640 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-13 00:54:13,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure2.ProcedureExecutor(1029): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:13,642 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 00:54:13,642 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-02-13 00:54:13,643 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 00:54:13,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-13 00:54:13,645 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8 2023-02-13 00:54:13,645 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8 empty. 2023-02-13 00:54:13,646 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8 2023-02-13 00:54:13,646 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-02-13 00:54:13,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-13 00:54:13,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-13 00:54:14,056 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-13 00:54:14,057 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 031fe1d6fc30ca635326f901d80077b8, NAME => 'testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp 2023-02-13 00:54:14,247 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-13 00:54:14,464 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:14,464 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1603): Closing 031fe1d6fc30ca635326f901d80077b8, disabling compactions & flushes 2023-02-13 00:54:14,464 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:14,464 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:14,464 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. after waiting 0 ms 2023-02-13 00:54:14,464 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:14,464 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1837): Closed testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:14,464 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 031fe1d6fc30ca635326f901d80077b8: Waiting for close lock at 1676249654464Disabling compacts and flushes for region at 1676249654464Disabling writes for close at 1676249654464Writing region close event to WAL at 1676249654464Closed at 1676249654464 2023-02-13 00:54:14,467 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 00:54:14,468 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676249654468"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249654468"}]},"ts":"1676249654468"} 2023-02-13 00:54:14,469 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 00:54:14,470 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 00:54:14,471 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249654470"}]},"ts":"1676249654470"} 2023-02-13 00:54:14,472 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-02-13 00:54:14,477 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=031fe1d6fc30ca635326f901d80077b8, ASSIGN}] 2023-02-13 00:54:14,478 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=031fe1d6fc30ca635326f901d80077b8, ASSIGN 2023-02-13 00:54:14,479 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=031fe1d6fc30ca635326f901d80077b8, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=false 2023-02-13 00:54:14,630 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=031fe1d6fc30ca635326f901d80077b8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:14,630 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676249654630"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249654630"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249654630"}]},"ts":"1676249654630"} 2023-02-13 00:54:14,632 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure 031fe1d6fc30ca635326f901d80077b8, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:54:14,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-13 00:54:14,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:14,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 031fe1d6fc30ca635326f901d80077b8, NAME => 'testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:54:14,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy 031fe1d6fc30ca635326f901d80077b8 2023-02-13 00:54:14,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:14,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 031fe1d6fc30ca635326f901d80077b8 2023-02-13 00:54:14,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 031fe1d6fc30ca635326f901d80077b8 2023-02-13 00:54:14,789 INFO [StoreOpener-031fe1d6fc30ca635326f901d80077b8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 031fe1d6fc30ca635326f901d80077b8 2023-02-13 00:54:14,791 DEBUG [StoreOpener-031fe1d6fc30ca635326f901d80077b8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1 2023-02-13 00:54:14,791 DEBUG [StoreOpener-031fe1d6fc30ca635326f901d80077b8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1 2023-02-13 00:54:14,791 INFO [StoreOpener-031fe1d6fc30ca635326f901d80077b8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 031fe1d6fc30ca635326f901d80077b8 columnFamilyName family1 2023-02-13 00:54:14,793 INFO [StoreOpener-031fe1d6fc30ca635326f901d80077b8-1] regionserver.HStore(310): Store=031fe1d6fc30ca635326f901d80077b8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:14,794 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8 2023-02-13 00:54:14,794 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8 2023-02-13 00:54:14,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 031fe1d6fc30ca635326f901d80077b8 2023-02-13 00:54:14,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:54:14,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 031fe1d6fc30ca635326f901d80077b8; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69490388, jitterRate=0.035487473011016846}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 00:54:14,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 031fe1d6fc30ca635326f901d80077b8: Running coprocessor pre-open hook at 1676249654788Writing region info on filesystem at 1676249654788Initializing all the Stores at 1676249654789 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676249654789Cleaning up temporary data from old regions at 1676249654795 (+6 ms)Cleaning up detritus from prior splits at 1676249654795Running coprocessor post-open hooks at 1676249654799 (+4 ms)Region opened successfully at 1676249654799 2023-02-13 00:54:14,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8., pid=25, masterSystemTime=1676249654784 2023-02-13 00:54:14,802 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:14,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:14,803 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=031fe1d6fc30ca635326f901d80077b8, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:14,803 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676249654803"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249654803"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249654803"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249654803"}]},"ts":"1676249654803"} 2023-02-13 00:54:14,806 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=24 2023-02-13 00:54:14,806 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure 031fe1d6fc30ca635326f901d80077b8, server=jenkins-hbase4.apache.org,36605,1676249622318 in 172 msec 2023-02-13 00:54:14,808 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-02-13 00:54:14,808 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=031fe1d6fc30ca635326f901d80077b8, ASSIGN in 329 msec 2023-02-13 00:54:14,809 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 00:54:14,809 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249654809"}]},"ts":"1676249654809"} 2023-02-13 00:54:14,810 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-02-13 00:54:14,813 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 00:54:14,815 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 1.1730 sec 2023-02-13 00:54:15,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-13 00:54:15,749 INFO [Listener at localhost/37869] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-02-13 00:54:15,774 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:15,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:15,776 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 00:54:15,776 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:15,776 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 00:54:15,776 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:15,777 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:15,777 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:15,778 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:15,778 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:15,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:15,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:15,779 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:15,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:15,779 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:15,779 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:15,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:15,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:15,780 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:15,780 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:15,780 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 00:54:15,780 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:15,780 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 00:54:15,780 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:15,781 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:15,781 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:15,781 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. started... 2023-02-13 00:54:15,781 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 031fe1d6fc30ca635326f901d80077b8 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-13 00:54:16,205 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130806cf0200f48402f803f371c1dbf0935 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151130806cf0200f48402f803f371c1dbf0935 2023-02-13 00:54:16,610 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/.tmp/family1/2193d8012ada4355a1b3ffd161d8d006 2023-02-13 00:54:16,616 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/.tmp/family1/2193d8012ada4355a1b3ffd161d8d006 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/2193d8012ada4355a1b3ffd161d8d006 2023-02-13 00:54:16,621 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/2193d8012ada4355a1b3ffd161d8d006, entries=1000, sequenceid=5, filesize=165.1 K 2023-02-13 00:54:16,622 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 031fe1d6fc30ca635326f901d80077b8 in 841ms, sequenceid=5, compaction requested=false 2023-02-13 00:54:16,622 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 00:54:16,623 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 031fe1d6fc30ca635326f901d80077b8: 2023-02-13 00:54:16,623 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:16,623 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:16,623 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:16,623 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:16,623 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 00:54:16,623 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 00:54:16,625 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:16,625 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:16,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:16,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:16,625 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:16,625 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:16,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:16,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:16,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:16,626 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 00:54:16,626 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2f1a3a40[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:16,626 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:16,626 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,628 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,628 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:16,628 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 00:54:16,628 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:16,628 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:16,628 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:16,629 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:16,629 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 00:54:16,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:16,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:16,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:16,629 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:16,630 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 00:54:16,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:16,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:16,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:16,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:16,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:16,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:16,632 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:16,632 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 00:54:16,632 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:16,632 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:16,632 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,632 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:16,635 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,635 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,635 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:16,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:16,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:16,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:16,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:16,635 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:16,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:16,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:16,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,636 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:16,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:16,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:16,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:16,638 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:16,638 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:16,638 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:16,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 00:54:16,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:16,639 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:16,639 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 00:54:16,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:16,639 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:16,639 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,639 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:16,639 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:16,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:16,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:16,639 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:16,639 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,639 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,639 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,739 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:16,740 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:16,745 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:16,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:16,747 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,747 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 00:54:16,747 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:16,747 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 00:54:16,747 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:16,747 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,747 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,749 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:16,749 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:16,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:16,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:16,749 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:16,749 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:16,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,749 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:16,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,750 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,750 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 00:54:16,750 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:16,750 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 00:54:16,750 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:16,750 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 00:54:16,751 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:16,751 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:16,751 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:16,751 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. started... 2023-02-13 00:54:16,751 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 031fe1d6fc30ca635326f901d80077b8 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-13 00:54:17,164 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128b96ad7a76adf4345889eb0eae39793f5 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151128b96ad7a76adf4345889eb0eae39793f5 2023-02-13 00:54:17,570 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/.tmp/family1/8b903054b2424ba58fa6da294357cff3 2023-02-13 00:54:17,577 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/.tmp/family1/8b903054b2424ba58fa6da294357cff3 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/8b903054b2424ba58fa6da294357cff3 2023-02-13 00:54:17,583 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/8b903054b2424ba58fa6da294357cff3, entries=1, sequenceid=9, filesize=5.0 K 2023-02-13 00:54:17,584 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 031fe1d6fc30ca635326f901d80077b8 in 833ms, sequenceid=9, compaction requested=false 2023-02-13 00:54:17,584 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 031fe1d6fc30ca635326f901d80077b8: 2023-02-13 00:54:17,584 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:17,584 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:17,584 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:17,584 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,584 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 00:54:17,584 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 00:54:17,586 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,586 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:17,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:17,586 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,586 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:17,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:17,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:17,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:17,588 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 00:54:17,588 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@291bbd2f[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:17,588 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:17,588 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,592 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,592 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:17,592 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 00:54:17,592 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,592 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:17,592 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:17,594 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,594 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 00:54:17,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:17,594 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:17,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:17,594 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 00:54:17,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:17,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:17,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:17,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:17,597 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:17,597 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 00:54:17,597 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:17,597 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:17,597 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,597 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:17,599 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,599 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:17,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:17,599 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:17,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:17,599 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:17,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:17,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:17,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,601 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:17,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,603 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,603 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,603 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:17,603 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:17,603 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 00:54:17,603 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 00:54:17,603 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:17,603 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,603 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:17,603 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:17,604 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,604 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,604 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,604 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:17,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:17,604 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:17,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:17,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:17,704 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:17,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:17,710 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:17,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:17,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,711 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 00:54:17,711 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:17,712 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 00:54:17,712 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:17,712 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,712 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,715 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,715 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:17,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:17,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:17,715 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,715 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:17,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,715 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:17,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,716 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,716 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 00:54:17,716 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,716 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 00:54:17,716 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:17,716 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 00:54:17,716 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:17,716 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:17,716 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:17,716 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. started... 2023-02-13 00:54:17,716 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 031fe1d6fc30ca635326f901d80077b8 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:17,731 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120139763b964d5c457ba6e62f914bd8cb59 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120139763b964d5c457ba6e62f914bd8cb59 2023-02-13 00:54:17,736 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/.tmp/family1/dd2000387cf745959aa313931737387a 2023-02-13 00:54:17,742 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/.tmp/family1/dd2000387cf745959aa313931737387a as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/dd2000387cf745959aa313931737387a 2023-02-13 00:54:17,747 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/dd2000387cf745959aa313931737387a, entries=1, sequenceid=13, filesize=4.9 K 2023-02-13 00:54:17,748 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 031fe1d6fc30ca635326f901d80077b8 in 32ms, sequenceid=13, compaction requested=true 2023-02-13 00:54:17,749 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 031fe1d6fc30ca635326f901d80077b8: 2023-02-13 00:54:17,749 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:17,749 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:17,749 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:17,749 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,749 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 00:54:17,749 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 00:54:17,751 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,751 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:17,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:17,751 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,751 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:17,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:17,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:17,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:17,753 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 00:54:17,753 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@336f316f[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:17,753 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:17,753 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,754 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,754 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:17,754 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 00:54:17,754 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,754 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:17,754 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:17,756 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,756 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 00:54:17,756 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:17,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:17,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:17,756 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 00:54:17,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:17,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:17,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:17,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:17,759 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:17,759 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 00:54:17,759 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:17,759 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:17,759 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,759 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:17,761 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,761 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:17,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:17,761 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:17,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,761 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:17,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:17,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:17,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:17,763 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:17,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,767 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,767 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,767 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:17,767 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:17,767 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 00:54:17,768 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:17,767 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:17,767 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:17,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:17,768 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 00:54:17,768 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,768 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:17,768 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:17,768 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:17,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:17,768 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,768 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,868 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:17,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:17,874 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:17,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:17,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,875 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 00:54:17,875 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:17,875 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 00:54:17,875 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:17,876 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,876 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,878 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,878 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:17,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:17,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:17,878 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:17,878 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:17,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,878 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:17,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,879 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,879 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 00:54:17,879 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:17,879 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 00:54:17,879 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:17,879 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 00:54:17,880 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:17,880 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:17,880 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:17,880 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. started... 2023-02-13 00:54:17,880 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 031fe1d6fc30ca635326f901d80077b8 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:18,295 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512051e4a0811a0f548c587567acee0cbe095 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512051e4a0811a0f548c587567acee0cbe095 2023-02-13 00:54:18,701 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/.tmp/family1/86553fb0311f45e5b3608799287949ab 2023-02-13 00:54:18,707 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/.tmp/family1/86553fb0311f45e5b3608799287949ab as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/86553fb0311f45e5b3608799287949ab 2023-02-13 00:54:18,712 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/86553fb0311f45e5b3608799287949ab, entries=1, sequenceid=17, filesize=4.9 K 2023-02-13 00:54:18,713 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 031fe1d6fc30ca635326f901d80077b8 in 833ms, sequenceid=17, compaction requested=true 2023-02-13 00:54:18,713 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 031fe1d6fc30ca635326f901d80077b8: 2023-02-13 00:54:18,714 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:18,714 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:18,714 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:18,714 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:18,714 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 00:54:18,714 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 00:54:18,716 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:18,716 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:18,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:18,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:18,717 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,717 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:18,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:18,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:18,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:18,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:18,718 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 00:54:18,718 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@54ac3779[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:18,718 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:18,719 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,720 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,720 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:18,720 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 00:54:18,720 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:18,720 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:18,720 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:18,721 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:18,721 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 00:54:18,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:18,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:18,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:18,721 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:18,722 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 00:54:18,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:18,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:18,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:18,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:18,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:18,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:18,724 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:18,724 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 00:54:18,724 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:18,724 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:18,724 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,724 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:18,726 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,726 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:18,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:18,726 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:18,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,726 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:18,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:18,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:18,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:18,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:18,728 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:18,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:18,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:18,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,731 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:18,731 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:18,731 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,731 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:18,731 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 00:54:18,731 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:18,731 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,731 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:18,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:18,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:18,731 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 00:54:18,731 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:18,732 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:18,732 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,732 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:18,732 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:18,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:18,732 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,832 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:18,833 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:18,838 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:18,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:18,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,840 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 00:54:18,840 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:18,840 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 00:54:18,840 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:18,841 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,841 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,842 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:18,842 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:18,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:18,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:18,842 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:18,842 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:18,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,843 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:18,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,843 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,843 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 00:54:18,843 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:18,843 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 00:54:18,843 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:18,843 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 00:54:18,844 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:18,844 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:18,844 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:18,844 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. started... 2023-02-13 00:54:18,844 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 031fe1d6fc30ca635326f901d80077b8 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:19,259 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122839e113c053fb4d338f443a179b5ef032 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015122839e113c053fb4d338f443a179b5ef032 2023-02-13 00:54:19,666 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/.tmp/family1/17e4442d3a79427192e9d067a25995ad 2023-02-13 00:54:19,673 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/.tmp/family1/17e4442d3a79427192e9d067a25995ad as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/17e4442d3a79427192e9d067a25995ad 2023-02-13 00:54:19,679 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/17e4442d3a79427192e9d067a25995ad, entries=1, sequenceid=21, filesize=4.9 K 2023-02-13 00:54:19,680 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 031fe1d6fc30ca635326f901d80077b8 in 836ms, sequenceid=21, compaction requested=true 2023-02-13 00:54:19,681 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 031fe1d6fc30ca635326f901d80077b8: 2023-02-13 00:54:19,681 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:19,681 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:19,681 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:19,681 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:19,681 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 00:54:19,681 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 00:54:19,683 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:19,683 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:19,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:19,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:19,684 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,684 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:19,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:19,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:19,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:19,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:19,686 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 00:54:19,686 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2a9adf96[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:19,686 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:19,686 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,687 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,688 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:19,688 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 00:54:19,688 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:19,688 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:19,688 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:19,690 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 00:54:19,690 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:19,690 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:19,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:19,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:19,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:19,690 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 00:54:19,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:19,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:19,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:19,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:19,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:19,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:19,694 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:19,694 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 00:54:19,694 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:19,694 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:19,694 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,694 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:19,696 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,696 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,696 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:19,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:19,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:19,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:19,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:19,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,697 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:19,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:19,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:19,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,699 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:19,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:19,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:19,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:19,703 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:19,703 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:19,703 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:19,703 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 00:54:19,703 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:19,703 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,703 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 00:54:19,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:19,703 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:19,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:19,703 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:19,703 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,704 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:19,704 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:19,704 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,704 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:19,704 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:19,804 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:19,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:19,810 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:19,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:19,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,817 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 00:54:19,817 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:19,818 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 00:54:19,818 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:19,818 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,818 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,820 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:19,820 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:19,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:19,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:19,820 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:19,820 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:19,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,821 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:19,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,821 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,822 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 00:54:19,822 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:19,822 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 00:54:19,822 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:19,822 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 00:54:19,822 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:19,822 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:19,822 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:19,822 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. started... 2023-02-13 00:54:19,822 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 031fe1d6fc30ca635326f901d80077b8 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:20,246 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231bb401160a22a46c095d413c3cf7b1c2a to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231bb401160a22a46c095d413c3cf7b1c2a 2023-02-13 00:54:20,334 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 00:54:20,373 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 00:54:20,374 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x10c3ee4b to 127.0.0.1:54478 2023-02-13 00:54:20,374 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:54:20,374 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 00:54:20,375 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5273cd1c to 127.0.0.1:54478 2023-02-13 00:54:20,375 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:54:20,375 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 00:54:20,375 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3a7fa659 to 127.0.0.1:54478 2023-02-13 00:54:20,375 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:54:20,652 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/.tmp/family1/bcb98fcc40c84749bd3578ef9669669a 2023-02-13 00:54:20,657 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/.tmp/family1/bcb98fcc40c84749bd3578ef9669669a as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/bcb98fcc40c84749bd3578ef9669669a 2023-02-13 00:54:20,661 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/bcb98fcc40c84749bd3578ef9669669a, entries=1, sequenceid=25, filesize=4.9 K 2023-02-13 00:54:20,662 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 031fe1d6fc30ca635326f901d80077b8 in 840ms, sequenceid=25, compaction requested=true 2023-02-13 00:54:20,662 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 031fe1d6fc30ca635326f901d80077b8: 2023-02-13 00:54:20,663 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:20,663 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:20,663 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:20,663 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:20,663 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 00:54:20,663 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 00:54:20,664 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:20,664 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:20,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:20,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:20,665 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,665 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:20,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:20,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:20,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:20,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:20,666 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 00:54:20,666 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5e9ced2f[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:20,666 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:20,666 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,667 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,668 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:20,668 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 00:54:20,668 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:20,668 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:20,668 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:20,675 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 00:54:20,675 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:20,675 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:20,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:20,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:20,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:20,675 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 00:54:20,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:20,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:20,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:20,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:20,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:20,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:20,677 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:20,677 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 00:54:20,677 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:20,677 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:20,677 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,677 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:20,679 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,679 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:20,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:20,679 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:20,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:20,679 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:20,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:20,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:20,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:20,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,681 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:20,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:20,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:20,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:20,683 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:20,683 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:20,683 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,683 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:20,683 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 00:54:20,683 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 00:54:20,684 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:20,683 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:20,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:20,683 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:20,684 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:20,684 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:20,684 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,684 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:20,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:20,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:20,684 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,684 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,784 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:20,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:20,789 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:20,791 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:20,791 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,791 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 00:54:20,791 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:20,791 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 00:54:20,791 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:20,792 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,792 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,793 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:20,793 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:20,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:20,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:20,793 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:20,793 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:20,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,794 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:20,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,794 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,794 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 00:54:20,794 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:20,794 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 00:54:20,795 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:20,795 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 00:54:20,796 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:20,796 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:20,796 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:20,796 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. started... 2023-02-13 00:54:20,796 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 031fe1d6fc30ca635326f901d80077b8 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:21,210 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601014d65cd83bbe84decab180ecd03078916 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601014d65cd83bbe84decab180ecd03078916 2023-02-13 00:54:21,616 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/.tmp/family1/1121c04d3b9c490c911ca85c319bc4d6 2023-02-13 00:54:21,621 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/.tmp/family1/1121c04d3b9c490c911ca85c319bc4d6 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/1121c04d3b9c490c911ca85c319bc4d6 2023-02-13 00:54:21,626 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/1121c04d3b9c490c911ca85c319bc4d6, entries=1, sequenceid=29, filesize=4.9 K 2023-02-13 00:54:21,627 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 031fe1d6fc30ca635326f901d80077b8 in 831ms, sequenceid=29, compaction requested=true 2023-02-13 00:54:21,627 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 031fe1d6fc30ca635326f901d80077b8: 2023-02-13 00:54:21,627 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:21,627 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:21,627 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:21,627 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:21,627 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 00:54:21,627 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 00:54:21,629 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:21,629 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:21,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:21,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:21,630 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,630 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:21,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:21,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:21,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:21,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:21,631 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 00:54:21,631 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@14b25bda[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:21,631 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:21,631 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,633 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,633 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:21,633 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:21,633 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:21,633 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 00:54:21,633 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:21,634 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:21,634 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 00:54:21,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:21,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:21,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:21,634 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:21,634 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 00:54:21,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:21,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:21,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:21,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:21,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:21,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:21,636 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:21,636 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 00:54:21,636 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:21,636 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:21,636 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,636 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:21,638 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,638 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:21,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:21,638 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:21,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:21,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:21,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:21,638 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:21,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:21,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:21,640 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:21,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:21,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:21,646 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:21,646 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:21,646 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:21,646 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:21,646 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 00:54:21,646 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:21,646 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:21,646 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:21,646 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:21,646 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:21,646 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,647 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:21,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:21,647 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:21,647 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,747 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:21,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:21,752 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:21,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:21,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,754 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 00:54:21,754 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:21,754 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 00:54:21,754 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:21,754 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,754 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,755 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:21,755 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:21,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:21,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:21,756 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:21,756 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:21,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,756 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:21,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,757 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,757 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 00:54:21,757 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:21,757 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 00:54:21,757 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:21,757 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 00:54:21,757 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:21,757 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:21,757 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:21,757 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. started... 2023-02-13 00:54:21,757 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 031fe1d6fc30ca635326f901d80077b8 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:21,773 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601037ea566ee5d8d4217bad60b2fc1132d90 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601037ea566ee5d8d4217bad60b2fc1132d90 2023-02-13 00:54:22,179 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/.tmp/family1/9b7116717c754508807e784618cd697c 2023-02-13 00:54:22,185 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/.tmp/family1/9b7116717c754508807e784618cd697c as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/9b7116717c754508807e784618cd697c 2023-02-13 00:54:22,189 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/9b7116717c754508807e784618cd697c, entries=1, sequenceid=33, filesize=4.9 K 2023-02-13 00:54:22,190 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 031fe1d6fc30ca635326f901d80077b8 in 433ms, sequenceid=33, compaction requested=true 2023-02-13 00:54:22,190 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 031fe1d6fc30ca635326f901d80077b8: 2023-02-13 00:54:22,190 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:22,190 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:22,190 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:22,190 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:22,190 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 00:54:22,190 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 00:54:22,192 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:22,192 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:22,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:22,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:22,192 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,192 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:22,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:22,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:22,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:22,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:22,194 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 00:54:22,194 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2110b1ca[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:22,194 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:22,194 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,195 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,195 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:22,196 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 00:54:22,195 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:22,196 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:22,196 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:22,200 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 00:54:22,200 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:22,200 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:22,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:22,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:22,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:22,200 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 00:54:22,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:22,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:22,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:22,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:22,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:22,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:22,203 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:22,203 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 00:54:22,203 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:22,203 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:22,203 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,203 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:22,204 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,204 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:22,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:22,204 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:22,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,205 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:22,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:22,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:22,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:22,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:22,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,206 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:22,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:22,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:22,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:22,208 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:22,208 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,208 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:22,208 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,208 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:22,208 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:22,208 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:22,208 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,208 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,208 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:22,208 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 00:54:22,208 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 00:54:22,208 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:22,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:22,208 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:22,209 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:22,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:22,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:22,309 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:22,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:22,314 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:22,315 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:22,315 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,315 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 00:54:22,315 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:22,316 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 00:54:22,316 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:22,316 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,316 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,317 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:22,317 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:22,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:22,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:22,318 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:22,318 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:22,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,318 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:22,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,318 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,319 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 00:54:22,319 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:22,319 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 00:54:22,319 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:22,319 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 00:54:22,319 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:22,319 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:22,319 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:22,319 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. started... 2023-02-13 00:54:22,319 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 031fe1d6fc30ca635326f901d80077b8 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:22,735 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090743c5db4010cb4ffeb23e3d467fcac9a8 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015090743c5db4010cb4ffeb23e3d467fcac9a8 2023-02-13 00:54:23,141 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/.tmp/family1/3b066e3a1899432f8b8d30ed8dae3a53 2023-02-13 00:54:23,146 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/.tmp/family1/3b066e3a1899432f8b8d30ed8dae3a53 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/3b066e3a1899432f8b8d30ed8dae3a53 2023-02-13 00:54:23,150 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/3b066e3a1899432f8b8d30ed8dae3a53, entries=1, sequenceid=37, filesize=4.9 K 2023-02-13 00:54:23,151 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 031fe1d6fc30ca635326f901d80077b8 in 832ms, sequenceid=37, compaction requested=true 2023-02-13 00:54:23,151 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 031fe1d6fc30ca635326f901d80077b8: 2023-02-13 00:54:23,151 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:23,151 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:23,151 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:23,151 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:23,151 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 00:54:23,151 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 00:54:23,154 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:23,154 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:23,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:23,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:23,154 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,154 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:23,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:23,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:23,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:23,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:23,155 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 00:54:23,155 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5d8aac11[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:23,155 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:23,155 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,159 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,160 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:23,160 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 00:54:23,160 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:23,160 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:23,160 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:23,161 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 00:54:23,161 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:23,161 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:23,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:23,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:23,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:23,161 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 00:54:23,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:23,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:23,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:23,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:23,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:23,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:23,163 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:23,163 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 00:54:23,163 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:23,163 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:23,163 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,163 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:23,169 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,169 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:23,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:23,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,169 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:23,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:23,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:23,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:23,170 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:23,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:23,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,171 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:23,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:23,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:23,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:23,173 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:23,173 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:23,173 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:23,173 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 00:54:23,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:23,173 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 00:54:23,173 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:23,173 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:23,173 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,173 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:23,174 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:23,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:23,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:23,174 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,174 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:23,174 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,174 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:23,174 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,274 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:23,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:23,279 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:23,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:23,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,280 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 00:54:23,281 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:23,281 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 00:54:23,281 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:23,281 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,281 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,282 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:23,282 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:23,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:23,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:23,283 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:23,283 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:23,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,283 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:23,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,283 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,284 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 00:54:23,284 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:23,284 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 00:54:23,284 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:23,284 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 00:54:23,284 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:23,284 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:23,284 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:23,284 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. started... 2023-02-13 00:54:23,284 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 031fe1d6fc30ca635326f901d80077b8 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:23,698 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511209747222c57e2463cbf27b1e171e24ae8 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201511209747222c57e2463cbf27b1e171e24ae8 2023-02-13 00:54:24,103 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/.tmp/family1/2046e2a93d034db5bd72d1149dc7fe92 2023-02-13 00:54:24,108 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/.tmp/family1/2046e2a93d034db5bd72d1149dc7fe92 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/2046e2a93d034db5bd72d1149dc7fe92 2023-02-13 00:54:24,113 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/2046e2a93d034db5bd72d1149dc7fe92, entries=1, sequenceid=41, filesize=4.9 K 2023-02-13 00:54:24,114 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 031fe1d6fc30ca635326f901d80077b8 in 830ms, sequenceid=41, compaction requested=true 2023-02-13 00:54:24,114 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 031fe1d6fc30ca635326f901d80077b8: 2023-02-13 00:54:24,114 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:24,114 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:24,114 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:24,114 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:24,114 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 00:54:24,114 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 00:54:24,116 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:24,116 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:24,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:24,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:24,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:24,116 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:24,116 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:24,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:24,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:24,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:24,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:24,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:24,117 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 00:54:24,117 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6446c71a[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:24,117 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:24,118 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:24,120 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:24,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:24,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:24,120 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:24,120 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 00:54:24,120 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:24,120 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:24,120 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:24,121 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 00:54:24,121 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:24,121 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:24,121 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 00:54:24,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:24,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:24,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:24,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:24,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:24,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:24,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:24,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:24,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:24,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:24,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:24,124 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:24,124 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 00:54:24,124 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:24,124 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:24,124 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:24,124 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:24,126 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:24,126 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:24,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:24,126 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:24,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:24,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:24,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:24,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:24,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:24,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:24,126 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:24,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:24,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:24,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:24,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:24,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:24,128 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:24,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:24,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:24,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:24,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:24,130 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:24,130 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:24,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:24,130 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 00:54:24,130 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:24,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:24,130 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:24,130 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 00:54:24,130 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:24,130 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:24,130 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:24,130 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:24,131 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:24,130 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:24,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:24,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:24,131 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:24,131 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:24,131 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:24,231 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:24,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:24,233 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43563] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-02-13 00:54:24,234 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] procedure2.ProcedureExecutor(1029): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 00:54:24,235 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 00:54:24,236 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:54:24,236 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676250264236 2023-02-13 00:54:24,237 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-13 00:54:24,238 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-02-13 00:54:24,238 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 00:54:24,239 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] zookeeper.ReadOnlyZKClient(139): Connect 0x6d907849 to 127.0.0.1:54478 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:54:24,244 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@53cd2b35, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:54:24,244 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 00:54:25,084 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103ed23e481c8b140a5a8e962cac8824e8a to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103ed23e481c8b140a5a8e962cac8824e8a 2023-02-13 00:54:25,085 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 00:54:25,097 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:54:25,098 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47178, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:54:25,104 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 00:54:25,106 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60474, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 00:54:25,115 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/4822ad9389d349a48ed41c98ad72ac65 first=Optional[r3] last=Optional[r6] 2023-02-13 00:54:25,115 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/4822ad9389d349a48ed41c98ad72ac65}] 2023-02-13 00:54:25,119 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/4822ad9389d349a48ed41c98ad72ac65 for inclusion in 031fe1d6fc30ca635326f901d80077b8/family1 2023-02-13 00:54:25,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-13 00:54:25,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 00:54:25,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for 031fe1d6fc30ca635326f901d80077b8: 2023-02-13 00:54:25,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/4822ad9389d349a48ed41c98ad72ac65 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMinorCompactionWithWeeklyPolicy__jfsgd14esq3c125rqln4iqks3p22a5dkuj0luqqb7vg7p4illisk13avkhci99mh/family1/4822ad9389d349a48ed41c98ad72ac65 2023-02-13 00:54:25,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMinorCompactionWithWeeklyPolicy__jfsgd14esq3c125rqln4iqks3p22a5dkuj0luqqb7vg7p4illisk13avkhci99mh/family1/4822ad9389d349a48ed41c98ad72ac65 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/4ece5e78b1f74924b1266008ebbd47a3_SeqId_44_ 2023-02-13 00:54:25,126 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMinorCompactionWithWeeklyPolicy__jfsgd14esq3c125rqln4iqks3p22a5dkuj0luqqb7vg7p4illisk13avkhci99mh/family1/4822ad9389d349a48ed41c98ad72ac65 into 031fe1d6fc30ca635326f901d80077b8/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/4ece5e78b1f74924b1266008ebbd47a3_SeqId_44_ - updating store file list. 2023-02-13 00:54:25,130 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/4ece5e78b1f74924b1266008ebbd47a3_SeqId_44_ into 031fe1d6fc30ca635326f901d80077b8/family1 2023-02-13 00:54:25,130 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMinorCompactionWithWeeklyPolicy__jfsgd14esq3c125rqln4iqks3p22a5dkuj0luqqb7vg7p4illisk13avkhci99mh/family1/4822ad9389d349a48ed41c98ad72ac65 into 031fe1d6fc30ca635326f901d80077b8/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/4ece5e78b1f74924b1266008ebbd47a3_SeqId_44_) 2023-02-13 00:54:25,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMinorCompactionWithWeeklyPolicy__jfsgd14esq3c125rqln4iqks3p22a5dkuj0luqqb7vg7p4illisk13avkhci99mh/family1/4822ad9389d349a48ed41c98ad72ac65 2023-02-13 00:54:25,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-13 00:54:25,134 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-13 00:54:25,134 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(1912): 031fe1d6fc30ca635326f901d80077b8/family1 is initiating minor compaction 2023-02-13 00:54:25,134 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1722a96c; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:25,134 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 031fe1d6fc30ca635326f901d80077b8/family1 in testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:54:25,134 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/8b903054b2424ba58fa6da294357cff3, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/dd2000387cf745959aa313931737387a, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/86553fb0311f45e5b3608799287949ab, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/17e4442d3a79427192e9d067a25995ad, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/bcb98fcc40c84749bd3578ef9669669a, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/1121c04d3b9c490c911ca85c319bc4d6, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/9b7116717c754508807e784618cd697c, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/3b066e3a1899432f8b8d30ed8dae3a53, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/2046e2a93d034db5bd72d1149dc7fe92, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/4ece5e78b1f74924b1266008ebbd47a3_SeqId_44_] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/.tmp, totalSize=50.0 K 2023-02-13 00:54:25,135 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 8b903054b2424ba58fa6da294357cff3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-13 00:54:25,135 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting dd2000387cf745959aa313931737387a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-13 00:54:25,135 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 86553fb0311f45e5b3608799287949ab, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-13 00:54:25,135 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 17e4442d3a79427192e9d067a25995ad, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-13 00:54:25,135 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting bcb98fcc40c84749bd3578ef9669669a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-13 00:54:25,135 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 1121c04d3b9c490c911ca85c319bc4d6, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-13 00:54:25,135 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 9b7116717c754508807e784618cd697c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-13 00:54:25,135 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 3b066e3a1899432f8b8d30ed8dae3a53, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-13 00:54:25,135 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 2046e2a93d034db5bd72d1149dc7fe92, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-13 00:54:25,135 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 4ece5e78b1f74924b1266008ebbd47a3_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-13 00:54:25,136 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 00:54:25,137 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:25,139 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015122839e113c053fb4d338f443a179b5ef032 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015122839e113c053fb4d338f443a179b5ef032 2023-02-13 00:54:25,140 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231bb401160a22a46c095d413c3cf7b1c2a to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231bb401160a22a46c095d413c3cf7b1c2a 2023-02-13 00:54:25,141 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601014d65cd83bbe84decab180ecd03078916 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601014d65cd83bbe84decab180ecd03078916 2023-02-13 00:54:25,142 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601037ea566ee5d8d4217bad60b2fc1132d90 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601037ea566ee5d8d4217bad60b2fc1132d90 2023-02-13 00:54:25,143 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-02-13 00:54:25,143 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-13 00:54:25,143 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 00:54:25,143 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 00:54:25,143 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 00:54:25,143 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 00:54:25,143 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 00:54:25,143 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 00:54:25,143 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 00:54:25,144 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 00:54:25,160 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] throttle.PressureAwareThroughputController(145): 031fe1d6fc30ca635326f901d80077b8#family1#compaction#34 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:54:25,575 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205c8d5742fa4ba48d1b52b813078d67785 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205c8d5742fa4ba48d1b52b813078d67785 2023-02-13 00:54:25,576 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 00:54:25,600 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/a72619e42ea9436e9cec672ca06eb531 first=Optional[r1] last=Optional[r2] 2023-02-13 00:54:25,601 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/a72619e42ea9436e9cec672ca06eb531}] 2023-02-13 00:54:25,604 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/a72619e42ea9436e9cec672ca06eb531 for inclusion in 031fe1d6fc30ca635326f901d80077b8/family1 2023-02-13 00:54:25,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-02-13 00:54:25,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 00:54:25,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for 031fe1d6fc30ca635326f901d80077b8: 2023-02-13 00:54:25,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/a72619e42ea9436e9cec672ca06eb531 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMinorCompactionWithWeeklyPolicy__7hoc8or7s2s754eqfk8q88bfahkff5553guijr5as0v35lb0gr2kbpe5ebld6iu/family1/a72619e42ea9436e9cec672ca06eb531 2023-02-13 00:54:25,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMinorCompactionWithWeeklyPolicy__7hoc8or7s2s754eqfk8q88bfahkff5553guijr5as0v35lb0gr2kbpe5ebld6iu/family1/a72619e42ea9436e9cec672ca06eb531 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/550d51e9430b46cd943c2aae9082f09f_SeqId_46_ 2023-02-13 00:54:25,611 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMinorCompactionWithWeeklyPolicy__7hoc8or7s2s754eqfk8q88bfahkff5553guijr5as0v35lb0gr2kbpe5ebld6iu/family1/a72619e42ea9436e9cec672ca06eb531 into 031fe1d6fc30ca635326f901d80077b8/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/550d51e9430b46cd943c2aae9082f09f_SeqId_46_ - updating store file list. 2023-02-13 00:54:25,615 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/550d51e9430b46cd943c2aae9082f09f_SeqId_46_ into 031fe1d6fc30ca635326f901d80077b8/family1 2023-02-13 00:54:25,615 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMinorCompactionWithWeeklyPolicy__7hoc8or7s2s754eqfk8q88bfahkff5553guijr5as0v35lb0gr2kbpe5ebld6iu/family1/a72619e42ea9436e9cec672ca06eb531 into 031fe1d6fc30ca635326f901d80077b8/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/550d51e9430b46cd943c2aae9082f09f_SeqId_46_) 2023-02-13 00:54:25,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMinorCompactionWithWeeklyPolicy__7hoc8or7s2s754eqfk8q88bfahkff5553guijr5as0v35lb0gr2kbpe5ebld6iu/family1/a72619e42ea9436e9cec672ca06eb531 2023-02-13 00:54:25,617 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 00:54:25,618 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:25,620 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120139763b964d5c457ba6e62f914bd8cb59 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120139763b964d5c457ba6e62f914bd8cb59 2023-02-13 00:54:25,620 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512051e4a0811a0f548c587567acee0cbe095 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512051e4a0811a0f548c587567acee0cbe095 2023-02-13 00:54:25,621 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-02-13 00:54:25,621 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-13 00:54:25,621 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 00:54:25,621 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 00:54:25,621 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:54:25,622 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 00:54:25,623 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.3890 sec 2023-02-13 00:54:25,725 INFO [Listener at localhost/37869] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=248 (was 257), OpenFileDescriptor=596 (was 615), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=216 (was 236), ProcessCount=168 (was 168), AvailableMemoryMB=4324 (was 4439) 2023-02-13 00:54:25,736 INFO [Listener at localhost/37869] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=248, OpenFileDescriptor=596, MaxFileDescriptor=60000, SystemLoadAverage=214, ProcessCount=168, AvailableMemoryMB=4324 2023-02-13 00:54:25,738 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-13 00:54:25,739 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:25,741 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 00:54:25,741 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-02-13 00:54:25,742 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 00:54:25,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-13 00:54:25,744 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testScannerOnBulkLoadRefHFiles/4ead7be1fbeb7ff33b9e4c0dd1135de8 2023-02-13 00:54:25,744 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testScannerOnBulkLoadRefHFiles/3fc81cadda520d51125d8aafce375f40 2023-02-13 00:54:25,744 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77 2023-02-13 00:54:25,744 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testScannerOnBulkLoadRefHFiles/4ead7be1fbeb7ff33b9e4c0dd1135de8 empty. 2023-02-13 00:54:25,744 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testScannerOnBulkLoadRefHFiles/3fc81cadda520d51125d8aafce375f40 empty. 2023-02-13 00:54:25,744 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77 empty. 2023-02-13 00:54:25,745 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77 2023-02-13 00:54:25,745 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testScannerOnBulkLoadRefHFiles/3fc81cadda520d51125d8aafce375f40 2023-02-13 00:54:25,745 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testScannerOnBulkLoadRefHFiles/4ead7be1fbeb7ff33b9e4c0dd1135de8 2023-02-13 00:54:25,745 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-02-13 00:54:25,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-13 00:54:25,979 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/.tmp/family1/3811635cd3f04eee9a0426a1e239a8c5 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/3811635cd3f04eee9a0426a1e239a8c5 2023-02-13 00:54:25,984 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 031fe1d6fc30ca635326f901d80077b8/family1 of 031fe1d6fc30ca635326f901d80077b8 into 3811635cd3f04eee9a0426a1e239a8c5(size=6.6 K), total size for store is 176.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:54:25,984 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 031fe1d6fc30ca635326f901d80077b8: 2023-02-13 00:54:25,985 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1676249665134; duration=0sec 2023-02-13 00:54:25,985 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:26,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-13 00:54:26,054 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/8b903054b2424ba58fa6da294357cff3, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/dd2000387cf745959aa313931737387a, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/86553fb0311f45e5b3608799287949ab, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/17e4442d3a79427192e9d067a25995ad, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/bcb98fcc40c84749bd3578ef9669669a, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/1121c04d3b9c490c911ca85c319bc4d6, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/9b7116717c754508807e784618cd697c, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/3b066e3a1899432f8b8d30ed8dae3a53, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/2046e2a93d034db5bd72d1149dc7fe92, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/4ece5e78b1f74924b1266008ebbd47a3_SeqId_44_] to archive 2023-02-13 00:54:26,055 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:26,057 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/8b903054b2424ba58fa6da294357cff3 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/8b903054b2424ba58fa6da294357cff3 2023-02-13 00:54:26,058 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/dd2000387cf745959aa313931737387a to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/dd2000387cf745959aa313931737387a 2023-02-13 00:54:26,060 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/86553fb0311f45e5b3608799287949ab to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/86553fb0311f45e5b3608799287949ab 2023-02-13 00:54:26,061 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/17e4442d3a79427192e9d067a25995ad to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/17e4442d3a79427192e9d067a25995ad 2023-02-13 00:54:26,062 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/bcb98fcc40c84749bd3578ef9669669a to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/bcb98fcc40c84749bd3578ef9669669a 2023-02-13 00:54:26,063 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/1121c04d3b9c490c911ca85c319bc4d6 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/1121c04d3b9c490c911ca85c319bc4d6 2023-02-13 00:54:26,064 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/9b7116717c754508807e784618cd697c to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/9b7116717c754508807e784618cd697c 2023-02-13 00:54:26,065 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/3b066e3a1899432f8b8d30ed8dae3a53 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/3b066e3a1899432f8b8d30ed8dae3a53 2023-02-13 00:54:26,066 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/2046e2a93d034db5bd72d1149dc7fe92 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/2046e2a93d034db5bd72d1149dc7fe92 2023-02-13 00:54:26,067 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/4ece5e78b1f74924b1266008ebbd47a3_SeqId_44_ to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/family1/4ece5e78b1f74924b1266008ebbd47a3_SeqId_44_ 2023-02-13 00:54:26,153 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-02-13 00:54:26,154 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7671): creating {ENCODED => 4ead7be1fbeb7ff33b9e4c0dd1135de8, NAME => 'testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp 2023-02-13 00:54:26,155 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7671): creating {ENCODED => 3fc81cadda520d51125d8aafce375f40, NAME => 'testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp 2023-02-13 00:54:26,155 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7671): creating {ENCODED => 086024b75418423ee9d686cb0fb63c77, NAME => 'testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp 2023-02-13 00:54:26,163 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:26,163 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1603): Closing 086024b75418423ee9d686cb0fb63c77, disabling compactions & flushes 2023-02-13 00:54:26,163 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77. 2023-02-13 00:54:26,163 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77. 2023-02-13 00:54:26,163 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77. after waiting 0 ms 2023-02-13 00:54:26,163 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77. 2023-02-13 00:54:26,163 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77. 2023-02-13 00:54:26,163 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1557): Region close journal for 086024b75418423ee9d686cb0fb63c77: Waiting for close lock at 1676249666163Disabling compacts and flushes for region at 1676249666163Disabling writes for close at 1676249666163Writing region close event to WAL at 1676249666163Closed at 1676249666163 2023-02-13 00:54:26,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-13 00:54:26,562 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:26,563 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1603): Closing 4ead7be1fbeb7ff33b9e4c0dd1135de8, disabling compactions & flushes 2023-02-13 00:54:26,563 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8. 2023-02-13 00:54:26,563 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8. 2023-02-13 00:54:26,563 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8. after waiting 0 ms 2023-02-13 00:54:26,563 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8. 2023-02-13 00:54:26,563 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8. 2023-02-13 00:54:26,563 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1557): Region close journal for 4ead7be1fbeb7ff33b9e4c0dd1135de8: Waiting for close lock at 1676249666563Disabling compacts and flushes for region at 1676249666563Disabling writes for close at 1676249666563Writing region close event to WAL at 1676249666563Closed at 1676249666563 2023-02-13 00:54:26,564 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:26,564 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1603): Closing 3fc81cadda520d51125d8aafce375f40, disabling compactions & flushes 2023-02-13 00:54:26,564 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40. 2023-02-13 00:54:26,564 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40. 2023-02-13 00:54:26,564 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40. after waiting 0 ms 2023-02-13 00:54:26,564 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40. 2023-02-13 00:54:26,564 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40. 2023-02-13 00:54:26,564 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1557): Region close journal for 3fc81cadda520d51125d8aafce375f40: Waiting for close lock at 1676249666564Disabling compacts and flushes for region at 1676249666564Disabling writes for close at 1676249666564Writing region close event to WAL at 1676249666564Closed at 1676249666564 2023-02-13 00:54:26,566 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 00:54:26,567 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676249666567"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249666567"}]},"ts":"1676249666567"} 2023-02-13 00:54:26,567 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676249666567"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249666567"}]},"ts":"1676249666567"} 2023-02-13 00:54:26,568 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676249666567"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249666567"}]},"ts":"1676249666567"} 2023-02-13 00:54:26,570 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-13 00:54:26,571 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 00:54:26,571 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249666571"}]},"ts":"1676249666571"} 2023-02-13 00:54:26,573 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-02-13 00:54:26,576 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4ead7be1fbeb7ff33b9e4c0dd1135de8, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=3fc81cadda520d51125d8aafce375f40, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=086024b75418423ee9d686cb0fb63c77, ASSIGN}] 2023-02-13 00:54:26,578 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=086024b75418423ee9d686cb0fb63c77, ASSIGN 2023-02-13 00:54:26,578 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=3fc81cadda520d51125d8aafce375f40, ASSIGN 2023-02-13 00:54:26,578 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4ead7be1fbeb7ff33b9e4c0dd1135de8, ASSIGN 2023-02-13 00:54:26,579 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=086024b75418423ee9d686cb0fb63c77, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=false 2023-02-13 00:54:26,579 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=3fc81cadda520d51125d8aafce375f40, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=false 2023-02-13 00:54:26,579 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4ead7be1fbeb7ff33b9e4c0dd1135de8, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=false 2023-02-13 00:54:26,730 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=3fc81cadda520d51125d8aafce375f40, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:26,730 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=086024b75418423ee9d686cb0fb63c77, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:26,730 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=4ead7be1fbeb7ff33b9e4c0dd1135de8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:26,730 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676249666730"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249666730"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249666730"}]},"ts":"1676249666730"} 2023-02-13 00:54:26,730 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676249666730"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249666730"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249666730"}]},"ts":"1676249666730"} 2023-02-13 00:54:26,730 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676249666730"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249666730"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249666730"}]},"ts":"1676249666730"} 2023-02-13 00:54:26,732 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=29, state=RUNNABLE; OpenRegionProcedure 3fc81cadda520d51125d8aafce375f40, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:54:26,733 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=32, ppid=30, state=RUNNABLE; OpenRegionProcedure 086024b75418423ee9d686cb0fb63c77, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:54:26,733 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=28, state=RUNNABLE; OpenRegionProcedure 4ead7be1fbeb7ff33b9e4c0dd1135de8, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:54:26,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-13 00:54:26,887 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8. 2023-02-13 00:54:26,887 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4ead7be1fbeb7ff33b9e4c0dd1135de8, NAME => 'testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8.', STARTKEY => '', ENDKEY => '1'} 2023-02-13 00:54:26,887 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 4ead7be1fbeb7ff33b9e4c0dd1135de8 2023-02-13 00:54:26,888 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:26,888 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 4ead7be1fbeb7ff33b9e4c0dd1135de8 2023-02-13 00:54:26,888 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 4ead7be1fbeb7ff33b9e4c0dd1135de8 2023-02-13 00:54:26,889 INFO [StoreOpener-4ead7be1fbeb7ff33b9e4c0dd1135de8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 4ead7be1fbeb7ff33b9e4c0dd1135de8 2023-02-13 00:54:26,890 DEBUG [StoreOpener-4ead7be1fbeb7ff33b9e4c0dd1135de8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/4ead7be1fbeb7ff33b9e4c0dd1135de8/family1 2023-02-13 00:54:26,890 DEBUG [StoreOpener-4ead7be1fbeb7ff33b9e4c0dd1135de8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/4ead7be1fbeb7ff33b9e4c0dd1135de8/family1 2023-02-13 00:54:26,891 INFO [StoreOpener-4ead7be1fbeb7ff33b9e4c0dd1135de8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4ead7be1fbeb7ff33b9e4c0dd1135de8 columnFamilyName family1 2023-02-13 00:54:26,892 INFO [StoreOpener-4ead7be1fbeb7ff33b9e4c0dd1135de8-1] regionserver.HStore(310): Store=4ead7be1fbeb7ff33b9e4c0dd1135de8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:26,892 INFO [StoreOpener-4ead7be1fbeb7ff33b9e4c0dd1135de8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 4ead7be1fbeb7ff33b9e4c0dd1135de8 2023-02-13 00:54:26,893 DEBUG [StoreOpener-4ead7be1fbeb7ff33b9e4c0dd1135de8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/4ead7be1fbeb7ff33b9e4c0dd1135de8/family2 2023-02-13 00:54:26,893 DEBUG [StoreOpener-4ead7be1fbeb7ff33b9e4c0dd1135de8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/4ead7be1fbeb7ff33b9e4c0dd1135de8/family2 2023-02-13 00:54:26,894 INFO [StoreOpener-4ead7be1fbeb7ff33b9e4c0dd1135de8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4ead7be1fbeb7ff33b9e4c0dd1135de8 columnFamilyName family2 2023-02-13 00:54:26,895 INFO [StoreOpener-4ead7be1fbeb7ff33b9e4c0dd1135de8-1] regionserver.HStore(310): Store=4ead7be1fbeb7ff33b9e4c0dd1135de8/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:26,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/4ead7be1fbeb7ff33b9e4c0dd1135de8 2023-02-13 00:54:26,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/4ead7be1fbeb7ff33b9e4c0dd1135de8 2023-02-13 00:54:26,897 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:54:26,898 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 4ead7be1fbeb7ff33b9e4c0dd1135de8 2023-02-13 00:54:26,900 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/4ead7be1fbeb7ff33b9e4c0dd1135de8/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:54:26,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 4ead7be1fbeb7ff33b9e4c0dd1135de8; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69656746, jitterRate=0.037966400384902954}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:54:26,901 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 4ead7be1fbeb7ff33b9e4c0dd1135de8: Running coprocessor pre-open hook at 1676249666888Writing region info on filesystem at 1676249666888Initializing all the Stores at 1676249666888Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249666888Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249666889 (+1 ms)Cleaning up temporary data from old regions at 1676249666897 (+8 ms)Cleaning up detritus from prior splits at 1676249666897Running coprocessor post-open hooks at 1676249666901 (+4 ms)Region opened successfully at 1676249666901 2023-02-13 00:54:26,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8., pid=33, masterSystemTime=1676249666884 2023-02-13 00:54:26,903 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8. 2023-02-13 00:54:26,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8. 2023-02-13 00:54:26,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40. 2023-02-13 00:54:26,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3fc81cadda520d51125d8aafce375f40, NAME => 'testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40.', STARTKEY => '1', ENDKEY => '2'} 2023-02-13 00:54:26,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 3fc81cadda520d51125d8aafce375f40 2023-02-13 00:54:26,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:26,904 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=4ead7be1fbeb7ff33b9e4c0dd1135de8, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:26,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 3fc81cadda520d51125d8aafce375f40 2023-02-13 00:54:26,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 3fc81cadda520d51125d8aafce375f40 2023-02-13 00:54:26,904 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676249666904"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249666904"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249666904"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249666904"}]},"ts":"1676249666904"} 2023-02-13 00:54:26,905 INFO [StoreOpener-3fc81cadda520d51125d8aafce375f40-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 3fc81cadda520d51125d8aafce375f40 2023-02-13 00:54:26,907 DEBUG [StoreOpener-3fc81cadda520d51125d8aafce375f40-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/3fc81cadda520d51125d8aafce375f40/family1 2023-02-13 00:54:26,907 DEBUG [StoreOpener-3fc81cadda520d51125d8aafce375f40-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/3fc81cadda520d51125d8aafce375f40/family1 2023-02-13 00:54:26,907 INFO [StoreOpener-3fc81cadda520d51125d8aafce375f40-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3fc81cadda520d51125d8aafce375f40 columnFamilyName family1 2023-02-13 00:54:26,908 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=28 2023-02-13 00:54:26,908 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=28, state=SUCCESS; OpenRegionProcedure 4ead7be1fbeb7ff33b9e4c0dd1135de8, server=jenkins-hbase4.apache.org,36605,1676249622318 in 173 msec 2023-02-13 00:54:26,908 INFO [StoreOpener-3fc81cadda520d51125d8aafce375f40-1] regionserver.HStore(310): Store=3fc81cadda520d51125d8aafce375f40/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:26,909 INFO [StoreOpener-3fc81cadda520d51125d8aafce375f40-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 3fc81cadda520d51125d8aafce375f40 2023-02-13 00:54:26,909 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4ead7be1fbeb7ff33b9e4c0dd1135de8, ASSIGN in 332 msec 2023-02-13 00:54:26,910 DEBUG [StoreOpener-3fc81cadda520d51125d8aafce375f40-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/3fc81cadda520d51125d8aafce375f40/family2 2023-02-13 00:54:26,910 DEBUG [StoreOpener-3fc81cadda520d51125d8aafce375f40-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/3fc81cadda520d51125d8aafce375f40/family2 2023-02-13 00:54:26,911 INFO [StoreOpener-3fc81cadda520d51125d8aafce375f40-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3fc81cadda520d51125d8aafce375f40 columnFamilyName family2 2023-02-13 00:54:26,912 INFO [StoreOpener-3fc81cadda520d51125d8aafce375f40-1] regionserver.HStore(310): Store=3fc81cadda520d51125d8aafce375f40/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:26,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/3fc81cadda520d51125d8aafce375f40 2023-02-13 00:54:26,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/3fc81cadda520d51125d8aafce375f40 2023-02-13 00:54:26,915 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:54:26,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 3fc81cadda520d51125d8aafce375f40 2023-02-13 00:54:26,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/3fc81cadda520d51125d8aafce375f40/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:54:26,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 3fc81cadda520d51125d8aafce375f40; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67043703, jitterRate=-9.709745645523071E-4}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:54:26,918 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 3fc81cadda520d51125d8aafce375f40: Running coprocessor pre-open hook at 1676249666904Writing region info on filesystem at 1676249666904Initializing all the Stores at 1676249666905 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249666905Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249666905Cleaning up temporary data from old regions at 1676249666914 (+9 ms)Cleaning up detritus from prior splits at 1676249666914Running coprocessor post-open hooks at 1676249666918 (+4 ms)Region opened successfully at 1676249666918 2023-02-13 00:54:26,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40., pid=31, masterSystemTime=1676249666884 2023-02-13 00:54:26,920 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40. 2023-02-13 00:54:26,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40. 2023-02-13 00:54:26,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77. 2023-02-13 00:54:26,920 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 086024b75418423ee9d686cb0fb63c77, NAME => 'testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77.', STARTKEY => '2', ENDKEY => ''} 2023-02-13 00:54:26,920 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 086024b75418423ee9d686cb0fb63c77 2023-02-13 00:54:26,920 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=3fc81cadda520d51125d8aafce375f40, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:26,920 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:26,920 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 086024b75418423ee9d686cb0fb63c77 2023-02-13 00:54:26,921 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 086024b75418423ee9d686cb0fb63c77 2023-02-13 00:54:26,921 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676249666920"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249666920"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249666920"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249666920"}]},"ts":"1676249666920"} 2023-02-13 00:54:26,922 INFO [StoreOpener-086024b75418423ee9d686cb0fb63c77-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 086024b75418423ee9d686cb0fb63c77 2023-02-13 00:54:26,923 DEBUG [StoreOpener-086024b75418423ee9d686cb0fb63c77-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/family1 2023-02-13 00:54:26,923 DEBUG [StoreOpener-086024b75418423ee9d686cb0fb63c77-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/family1 2023-02-13 00:54:26,924 INFO [StoreOpener-086024b75418423ee9d686cb0fb63c77-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 086024b75418423ee9d686cb0fb63c77 columnFamilyName family1 2023-02-13 00:54:26,924 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=29 2023-02-13 00:54:26,924 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=29, state=SUCCESS; OpenRegionProcedure 3fc81cadda520d51125d8aafce375f40, server=jenkins-hbase4.apache.org,36605,1676249622318 in 190 msec 2023-02-13 00:54:26,925 INFO [StoreOpener-086024b75418423ee9d686cb0fb63c77-1] regionserver.HStore(310): Store=086024b75418423ee9d686cb0fb63c77/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:26,925 INFO [StoreOpener-086024b75418423ee9d686cb0fb63c77-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 086024b75418423ee9d686cb0fb63c77 2023-02-13 00:54:26,925 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=3fc81cadda520d51125d8aafce375f40, ASSIGN in 348 msec 2023-02-13 00:54:26,926 DEBUG [StoreOpener-086024b75418423ee9d686cb0fb63c77-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/family2 2023-02-13 00:54:26,926 DEBUG [StoreOpener-086024b75418423ee9d686cb0fb63c77-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/family2 2023-02-13 00:54:26,926 INFO [StoreOpener-086024b75418423ee9d686cb0fb63c77-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 086024b75418423ee9d686cb0fb63c77 columnFamilyName family2 2023-02-13 00:54:26,927 INFO [StoreOpener-086024b75418423ee9d686cb0fb63c77-1] regionserver.HStore(310): Store=086024b75418423ee9d686cb0fb63c77/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:26,928 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77 2023-02-13 00:54:26,928 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77 2023-02-13 00:54:26,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:54:26,931 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 086024b75418423ee9d686cb0fb63c77 2023-02-13 00:54:26,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:54:26,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 086024b75418423ee9d686cb0fb63c77; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71282508, jitterRate=0.06219214200973511}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:54:26,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 086024b75418423ee9d686cb0fb63c77: Running coprocessor pre-open hook at 1676249666921Writing region info on filesystem at 1676249666921Initializing all the Stores at 1676249666921Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249666921Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249666922 (+1 ms)Cleaning up temporary data from old regions at 1676249666929 (+7 ms)Cleaning up detritus from prior splits at 1676249666929Running coprocessor post-open hooks at 1676249666934 (+5 ms)Region opened successfully at 1676249666934 2023-02-13 00:54:26,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77., pid=32, masterSystemTime=1676249666884 2023-02-13 00:54:26,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77. 2023-02-13 00:54:26,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77. 2023-02-13 00:54:26,937 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=086024b75418423ee9d686cb0fb63c77, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:26,937 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676249666937"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249666937"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249666937"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249666937"}]},"ts":"1676249666937"} 2023-02-13 00:54:26,940 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=30 2023-02-13 00:54:26,940 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=30, state=SUCCESS; OpenRegionProcedure 086024b75418423ee9d686cb0fb63c77, server=jenkins-hbase4.apache.org,36605,1676249622318 in 205 msec 2023-02-13 00:54:26,942 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=27 2023-02-13 00:54:26,942 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=086024b75418423ee9d686cb0fb63c77, ASSIGN in 364 msec 2023-02-13 00:54:26,942 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 00:54:26,943 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249666942"}]},"ts":"1676249666942"} 2023-02-13 00:54:26,944 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-02-13 00:54:26,946 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 00:54:26,948 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 1.2080 sec 2023-02-13 00:54:27,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-13 00:54:27,847 INFO [Listener at localhost/37869] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-02-13 00:54:27,853 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:27,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:27,855 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-02-13 00:54:27,855 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:27,855 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-02-13 00:54:27,855 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-02-13 00:54:27,856 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:27,856 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:27,858 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:27,858 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:27,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:27,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:27,859 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:27,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:27,859 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:27,859 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:27,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:27,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:27,859 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:27,860 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:27,860 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-02-13 00:54:27,860 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:27,860 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-02-13 00:54:27,860 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77. 2023-02-13 00:54:27,860 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 00:54:27,860 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:27,860 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40. 2023-02-13 00:54:27,860 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77. started... 2023-02-13 00:54:27,860 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40. started... 2023-02-13 00:54:27,860 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8. 2023-02-13 00:54:27,861 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8. started... 2023-02-13 00:54:27,861 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 086024b75418423ee9d686cb0fb63c77 2/2 column families, dataSize=48 B heapSize=632 B 2023-02-13 00:54:27,861 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 3fc81cadda520d51125d8aafce375f40: 2023-02-13 00:54:27,861 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 4ead7be1fbeb7ff33b9e4c0dd1135de8: 2023-02-13 00:54:27,861 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40. 2023-02-13 00:54:27,862 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8. 2023-02-13 00:54:27,862 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 00:54:27,862 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 00:54:28,274 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302134caff1b18e3346159e12381cf199c247 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202302134caff1b18e3346159e12381cf199c247 2023-02-13 00:54:28,679 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/.tmp/family1/6072a81d722244d78589416703d62d39 2023-02-13 00:54:28,684 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/.tmp/family1/6072a81d722244d78589416703d62d39 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/family1/6072a81d722244d78589416703d62d39 2023-02-13 00:54:28,689 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/family1/6072a81d722244d78589416703d62d39, entries=1, sequenceid=5, filesize=4.9 K 2023-02-13 00:54:28,690 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for 086024b75418423ee9d686cb0fb63c77 in 829ms, sequenceid=5, compaction requested=false 2023-02-13 00:54:28,690 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-02-13 00:54:28,691 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 086024b75418423ee9d686cb0fb63c77: 2023-02-13 00:54:28,691 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77. 2023-02-13 00:54:28,691 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 00:54:28,691 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 00:54:28,691 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:28,691 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-02-13 00:54:28,691 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-02-13 00:54:28,693 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,693 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:28,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:28,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:28,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:28,693 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,693 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:28,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:28,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:28,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:28,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:28,694 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-02-13 00:54:28,694 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@62ac2cc3[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:28,694 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-02-13 00:54:28,694 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,696 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,696 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-02-13 00:54:28,696 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-02-13 00:54:28,696 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:28,696 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:28,696 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:28,698 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:28,698 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-02-13 00:54:28,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:28,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:28,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:28,698 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:28,698 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-02-13 00:54:28,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:28,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:28,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:28,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:28,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:28,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:28,700 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-02-13 00:54:28,700 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-02-13 00:54:28,700 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:28,700 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:28,700 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,700 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:28,703 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,703 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:28,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:28,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,703 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:28,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:28,704 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:28,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:28,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:28,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:28,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:28,705 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:28,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:28,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:28,710 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:28,710 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:28,710 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:28,710 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-02-13 00:54:28,710 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:28,710 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:28,710 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,710 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:28,710 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:28,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:28,710 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,710 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:28,710 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:28,710 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,710 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:28,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:28,810 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:28,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:28,815 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:28,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:28,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,817 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-02-13 00:54:28,817 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:28,817 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-02-13 00:54:28,817 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-02-13 00:54:28,817 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,818 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,819 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:28,819 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:28,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:28,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:28,819 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:28,819 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:28,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,820 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:28,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,820 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,821 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-02-13 00:54:28,821 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:28,821 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-02-13 00:54:28,821 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:28,821 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-02-13 00:54:28,821 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77. 2023-02-13 00:54:28,821 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 00:54:28,821 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:28,821 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8. 2023-02-13 00:54:28,821 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77. started... 2023-02-13 00:54:28,821 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40. 2023-02-13 00:54:28,821 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8. started... 2023-02-13 00:54:28,821 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 086024b75418423ee9d686cb0fb63c77 2/2 column families, dataSize=90 B heapSize=752 B 2023-02-13 00:54:28,821 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40. started... 2023-02-13 00:54:28,821 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 4ead7be1fbeb7ff33b9e4c0dd1135de8: 2023-02-13 00:54:28,822 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8. 2023-02-13 00:54:28,822 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 3fc81cadda520d51125d8aafce375f40: 2023-02-13 00:54:28,822 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 00:54:28,822 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40. 2023-02-13 00:54:28,822 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 00:54:29,236 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021319f71f64b734470cb3dc757ec16f1393 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023021319f71f64b734470cb3dc757ec16f1393 2023-02-13 00:54:29,642 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/.tmp/family1/4c35a663d72e4604abc99613db491173 2023-02-13 00:54:29,647 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/.tmp/family1/4c35a663d72e4604abc99613db491173 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/family1/4c35a663d72e4604abc99613db491173 2023-02-13 00:54:29,651 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/family1/4c35a663d72e4604abc99613db491173, entries=2, sequenceid=9, filesize=5.0 K 2023-02-13 00:54:29,651 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for 086024b75418423ee9d686cb0fb63c77 in 830ms, sequenceid=9, compaction requested=false 2023-02-13 00:54:29,652 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 086024b75418423ee9d686cb0fb63c77: 2023-02-13 00:54:29,652 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77. 2023-02-13 00:54:29,652 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 00:54:29,652 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 00:54:29,652 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:29,652 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-02-13 00:54:29,652 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-02-13 00:54:29,654 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:29,654 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:29,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:29,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:29,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:29,654 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:29,654 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:29,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:29,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:29,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:29,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:29,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:29,656 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-02-13 00:54:29,656 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@46164fbd[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:29,656 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-02-13 00:54:29,656 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:29,657 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:29,657 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:29,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:29,657 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-02-13 00:54:29,657 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-02-13 00:54:29,657 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:29,657 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:29,657 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:29,658 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-02-13 00:54:29,658 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:29,658 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:29,658 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-02-13 00:54:29,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:29,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:29,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:29,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:29,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:29,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:29,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:29,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:29,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:29,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:29,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:29,661 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-02-13 00:54:29,661 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-02-13 00:54:29,661 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:29,661 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:29,661 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:29,661 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:29,662 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:29,662 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:29,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:29,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:29,662 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:29,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:29,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:29,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:29,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:29,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:29,663 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:29,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:29,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:29,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:29,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:29,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:29,664 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:29,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:29,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:29,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:29,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:29,671 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:29,671 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:29,671 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:29,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:29,672 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:29,672 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-02-13 00:54:29,671 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:29,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:29,672 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-02-13 00:54:29,672 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:29,672 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:29,672 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:29,672 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:29,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:29,672 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:29,672 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:29,672 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:29,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:29,672 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:29,772 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:29,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:29,780 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43563] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-02-13 00:54:29,781 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] procedure2.ProcedureExecutor(1029): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-13 00:54:29,782 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-13 00:54:29,782 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:54:29,782 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676250269782 2023-02-13 00:54:29,783 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 00:54:29,783 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-02-13 00:54:29,783 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-13 00:54:29,784 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] zookeeper.ReadOnlyZKClient(139): Connect 0x53f4ef3a to 127.0.0.1:54478 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:54:29,787 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@69dae2f2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:54:29,787 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230213 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-02-13 00:54:30,618 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213695ec8dd29a94421bb14b9e6569f7bdb to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230213695ec8dd29a94421bb14b9e6569f7bdb 2023-02-13 00:54:30,619 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-13 00:54:30,631 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:54:30,633 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47186, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:54:30,640 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 00:54:30,642 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60490, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 00:54:30,653 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230213/family1/c8de71dcff4643a7a3f1ca885960634d first=Optional[k0] last=Optional[k1] 2023-02-13 00:54:30,654 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230213/family1/c8de71dcff4643a7a3f1ca885960634d}] 2023-02-13 00:54:30,658 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230213/family1/c8de71dcff4643a7a3f1ca885960634d for inclusion in 086024b75418423ee9d686cb0fb63c77/family1 2023-02-13 00:54:30,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-02-13 00:54:30,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-13 00:54:30,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for 086024b75418423ee9d686cb0fb63c77: 2023-02-13 00:54:30,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230213/family1/c8de71dcff4643a7a3f1ca885960634d to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testScannerOnBulkLoadRefHFiles__qoghqnascao3gmd21mf5mb7qdevstissft7thblmg19o72fmfjrj76r7hq6veup3/family1/c8de71dcff4643a7a3f1ca885960634d 2023-02-13 00:54:30,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testScannerOnBulkLoadRefHFiles__qoghqnascao3gmd21mf5mb7qdevstissft7thblmg19o72fmfjrj76r7hq6veup3/family1/c8de71dcff4643a7a3f1ca885960634d as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/family1/f4c4b6a75e484ea5afd178097950f848_SeqId_12_ 2023-02-13 00:54:30,664 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testScannerOnBulkLoadRefHFiles__qoghqnascao3gmd21mf5mb7qdevstissft7thblmg19o72fmfjrj76r7hq6veup3/family1/c8de71dcff4643a7a3f1ca885960634d into 086024b75418423ee9d686cb0fb63c77/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/family1/f4c4b6a75e484ea5afd178097950f848_SeqId_12_ - updating store file list. 2023-02-13 00:54:30,667 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/family1/f4c4b6a75e484ea5afd178097950f848_SeqId_12_ into 086024b75418423ee9d686cb0fb63c77/family1 2023-02-13 00:54:30,667 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testScannerOnBulkLoadRefHFiles__qoghqnascao3gmd21mf5mb7qdevstissft7thblmg19o72fmfjrj76r7hq6veup3/family1/c8de71dcff4643a7a3f1ca885960634d into 086024b75418423ee9d686cb0fb63c77/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/family1/f4c4b6a75e484ea5afd178097950f848_SeqId_12_) 2023-02-13 00:54:30,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testScannerOnBulkLoadRefHFiles__qoghqnascao3gmd21mf5mb7qdevstissft7thblmg19o72fmfjrj76r7hq6veup3/family1/c8de71dcff4643a7a3f1ca885960634d 2023-02-13 00:54:30,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-13 00:54:30,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-13 00:54:30,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(1912): 086024b75418423ee9d686cb0fb63c77/family1 is initiating minor compaction (all files) 2023-02-13 00:54:30,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6e3e8d8f; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:30,669 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 086024b75418423ee9d686cb0fb63c77/family1 in testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77. 2023-02-13 00:54:30,669 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/family1/6072a81d722244d78589416703d62d39, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/family1/4c35a663d72e4604abc99613db491173, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/family1/f4c4b6a75e484ea5afd178097950f848_SeqId_12_] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/.tmp, totalSize=15.1 K 2023-02-13 00:54:30,669 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 6072a81d722244d78589416703d62d39, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676249667847 2023-02-13 00:54:30,670 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 4c35a663d72e4604abc99613db491173, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676249667847 2023-02-13 00:54:30,671 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-13 00:54:30,671 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting f4c4b6a75e484ea5afd178097950f848_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676249667847 2023-02-13 00:54:30,671 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:30,673 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023021319f71f64b734470cb3dc757ec16f1393 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023021319f71f64b734470cb3dc757ec16f1393 2023-02-13 00:54:30,674 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202302134caff1b18e3346159e12381cf199c247 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202302134caff1b18e3346159e12381cf199c247 2023-02-13 00:54:30,674 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-02-13 00:54:30,674 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230213' 2023-02-13 00:54:30,675 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230213 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-02-13 00:54:30,675 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-13 00:54:30,675 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:54:30,676 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-13 00:54:30,677 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 896 msec 2023-02-13 00:54:30,681 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] throttle.PressureAwareThroughputController(145): 086024b75418423ee9d686cb0fb63c77#family1#compaction#37 average throughput is 0.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:54:30,711 INFO [Listener at localhost/37869] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=247 (was 248), OpenFileDescriptor=590 (was 596), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=214 (was 214), ProcessCount=168 (was 168), AvailableMemoryMB=4319 (was 4324) 2023-02-13 00:54:30,721 INFO [Listener at localhost/37869] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=247, OpenFileDescriptor=590, MaxFileDescriptor=60000, SystemLoadAverage=214, ProcessCount=168, AvailableMemoryMB=4319 2023-02-13 00:54:30,723 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-13 00:54:30,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:30,725 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 00:54:30,725 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-02-13 00:54:30,726 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 00:54:30,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 00:54:30,727 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152 2023-02-13 00:54:30,727 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152 empty. 2023-02-13 00:54:30,728 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152 2023-02-13 00:54:30,728 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-02-13 00:54:30,827 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 00:54:31,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 00:54:31,135 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-13 00:54:31,136 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 893fe9ccf4b1790a755cd18beab22152, NAME => 'testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp 2023-02-13 00:54:31,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 00:54:31,542 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:31,542 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1603): Closing 893fe9ccf4b1790a755cd18beab22152, disabling compactions & flushes 2023-02-13 00:54:31,542 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:31,542 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:31,542 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. after waiting 0 ms 2023-02-13 00:54:31,542 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:31,542 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:31,542 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 893fe9ccf4b1790a755cd18beab22152: Waiting for close lock at 1676249671542Disabling compacts and flushes for region at 1676249671542Disabling writes for close at 1676249671542Writing region close event to WAL at 1676249671542Closed at 1676249671542 2023-02-13 00:54:31,544 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 00:54:31,545 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676249671545"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249671545"}]},"ts":"1676249671545"} 2023-02-13 00:54:31,546 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 00:54:31,547 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 00:54:31,547 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249671547"}]},"ts":"1676249671547"} 2023-02-13 00:54:31,548 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-02-13 00:54:31,551 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=893fe9ccf4b1790a755cd18beab22152, ASSIGN}] 2023-02-13 00:54:31,553 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=893fe9ccf4b1790a755cd18beab22152, ASSIGN 2023-02-13 00:54:31,553 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=893fe9ccf4b1790a755cd18beab22152, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=false 2023-02-13 00:54:31,704 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=893fe9ccf4b1790a755cd18beab22152, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:31,704 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676249671704"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249671704"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249671704"}]},"ts":"1676249671704"} 2023-02-13 00:54:31,706 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure 893fe9ccf4b1790a755cd18beab22152, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:54:31,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 00:54:31,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:31,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 893fe9ccf4b1790a755cd18beab22152, NAME => 'testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:54:31,861 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy 893fe9ccf4b1790a755cd18beab22152 2023-02-13 00:54:31,861 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:31,861 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 893fe9ccf4b1790a755cd18beab22152 2023-02-13 00:54:31,861 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 893fe9ccf4b1790a755cd18beab22152 2023-02-13 00:54:31,862 INFO [StoreOpener-893fe9ccf4b1790a755cd18beab22152-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 893fe9ccf4b1790a755cd18beab22152 2023-02-13 00:54:31,863 DEBUG [StoreOpener-893fe9ccf4b1790a755cd18beab22152-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1 2023-02-13 00:54:31,864 DEBUG [StoreOpener-893fe9ccf4b1790a755cd18beab22152-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1 2023-02-13 00:54:31,864 INFO [StoreOpener-893fe9ccf4b1790a755cd18beab22152-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 893fe9ccf4b1790a755cd18beab22152 columnFamilyName family1 2023-02-13 00:54:31,865 INFO [StoreOpener-893fe9ccf4b1790a755cd18beab22152-1] regionserver.HStore(310): Store=893fe9ccf4b1790a755cd18beab22152/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:31,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152 2023-02-13 00:54:31,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152 2023-02-13 00:54:31,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 893fe9ccf4b1790a755cd18beab22152 2023-02-13 00:54:31,870 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:54:31,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 893fe9ccf4b1790a755cd18beab22152; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60199226, jitterRate=-0.10296162962913513}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 00:54:31,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 893fe9ccf4b1790a755cd18beab22152: Running coprocessor pre-open hook at 1676249671861Writing region info on filesystem at 1676249671861Initializing all the Stores at 1676249671862 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676249671862Cleaning up temporary data from old regions at 1676249671867 (+5 ms)Cleaning up detritus from prior splits at 1676249671867Running coprocessor post-open hooks at 1676249671871 (+4 ms)Region opened successfully at 1676249671871 2023-02-13 00:54:31,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152., pid=37, masterSystemTime=1676249671857 2023-02-13 00:54:31,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:31,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:31,874 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=893fe9ccf4b1790a755cd18beab22152, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:31,874 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676249671874"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249671874"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249671874"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249671874"}]},"ts":"1676249671874"} 2023-02-13 00:54:31,876 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=36 2023-02-13 00:54:31,876 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure 893fe9ccf4b1790a755cd18beab22152, server=jenkins-hbase4.apache.org,36605,1676249622318 in 169 msec 2023-02-13 00:54:31,878 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-02-13 00:54:31,878 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=893fe9ccf4b1790a755cd18beab22152, ASSIGN in 325 msec 2023-02-13 00:54:31,878 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 00:54:31,878 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249671878"}]},"ts":"1676249671878"} 2023-02-13 00:54:31,879 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-02-13 00:54:31,882 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 00:54:31,883 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 1.1590 sec 2023-02-13 00:54:31,900 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/.tmp/family1/ce7a0478e19d4994877db4a3ca40d733 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/family1/ce7a0478e19d4994877db4a3ca40d733 2023-02-13 00:54:31,906 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 086024b75418423ee9d686cb0fb63c77/family1 of 086024b75418423ee9d686cb0fb63c77 into ce7a0478e19d4994877db4a3ca40d733(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 00:54:31,906 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 086024b75418423ee9d686cb0fb63c77: 2023-02-13 00:54:31,906 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1676249670669; duration=1sec 2023-02-13 00:54:31,906 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:31,948 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/family1/6072a81d722244d78589416703d62d39, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/family1/4c35a663d72e4604abc99613db491173, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/family1/f4c4b6a75e484ea5afd178097950f848_SeqId_12_] to archive 2023-02-13 00:54:31,949 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:31,950 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/family1/6072a81d722244d78589416703d62d39 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/family1/6072a81d722244d78589416703d62d39 2023-02-13 00:54:31,952 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/family1/4c35a663d72e4604abc99613db491173 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/family1/4c35a663d72e4604abc99613db491173 2023-02-13 00:54:31,953 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/family1/f4c4b6a75e484ea5afd178097950f848_SeqId_12_ to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/family1/f4c4b6a75e484ea5afd178097950f848_SeqId_12_ 2023-02-13 00:54:32,416 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 00:54:32,831 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 00:54:32,831 INFO [Listener at localhost/37869] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-02-13 00:54:32,853 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:32,854 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:32,854 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 00:54:32,854 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:32,855 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 00:54:32,855 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:32,855 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:32,855 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:32,856 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:32,856 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:32,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:32,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:32,857 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:32,857 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:32,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:32,857 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:32,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:32,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:32,858 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:32,858 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:32,858 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 00:54:32,858 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:32,858 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 00:54:32,858 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:32,858 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:32,858 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:32,858 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. started... 2023-02-13 00:54:32,859 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 893fe9ccf4b1790a755cd18beab22152 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-13 00:54:33,284 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113070b56723b4b6446d85379f61d8600cdc to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015113070b56723b4b6446d85379f61d8600cdc 2023-02-13 00:54:33,289 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/.tmp/family1/7cea6b39c5d54eed870857fbacd06638 2023-02-13 00:54:33,296 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/.tmp/family1/7cea6b39c5d54eed870857fbacd06638 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/7cea6b39c5d54eed870857fbacd06638 2023-02-13 00:54:33,300 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/7cea6b39c5d54eed870857fbacd06638, entries=1000, sequenceid=5, filesize=165.1 K 2023-02-13 00:54:33,301 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 893fe9ccf4b1790a755cd18beab22152 in 442ms, sequenceid=5, compaction requested=false 2023-02-13 00:54:33,301 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 00:54:33,302 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 893fe9ccf4b1790a755cd18beab22152: 2023-02-13 00:54:33,302 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:33,302 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:33,302 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:33,302 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,302 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 00:54:33,302 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 00:54:33,304 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,304 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:33,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:33,304 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,304 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:33,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:33,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:33,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:33,306 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 00:54:33,306 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4ef4994a[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:33,306 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:33,306 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,307 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,307 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:33,308 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 00:54:33,308 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,308 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:33,308 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:33,315 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,315 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 00:54:33,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:33,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:33,315 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:33,315 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 00:54:33,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:33,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:33,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:33,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:33,317 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:33,317 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 00:54:33,317 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:33,317 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:33,317 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,317 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:33,319 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,319 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:33,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:33,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,319 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:33,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:33,320 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:33,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:33,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:33,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,321 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:33,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,323 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,323 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,323 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,323 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:33,323 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:33,324 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:33,324 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,324 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:33,324 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,324 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 00:54:33,324 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:33,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:33,324 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,324 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:33,324 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:33,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:33,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:33,424 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:33,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:33,429 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:33,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:33,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,435 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 00:54:33,435 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:33,435 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 00:54:33,435 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:33,436 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,436 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,437 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,437 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:33,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:33,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:33,437 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,437 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:33,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,438 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:33,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,438 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,438 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 00:54:33,438 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,438 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 00:54:33,438 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:33,438 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 00:54:33,439 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:33,439 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:33,439 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:33,439 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. started... 2023-02-13 00:54:33,439 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 893fe9ccf4b1790a755cd18beab22152 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-13 00:54:33,855 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511282e7fce6ad0054ff381c76a58aa2e07b3 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511282e7fce6ad0054ff381c76a58aa2e07b3 2023-02-13 00:54:33,861 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/.tmp/family1/2aa13bbd06934c8f97c366f4c7a817d2 2023-02-13 00:54:33,866 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/.tmp/family1/2aa13bbd06934c8f97c366f4c7a817d2 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/2aa13bbd06934c8f97c366f4c7a817d2 2023-02-13 00:54:33,872 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/2aa13bbd06934c8f97c366f4c7a817d2, entries=1, sequenceid=9, filesize=5.0 K 2023-02-13 00:54:33,873 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 893fe9ccf4b1790a755cd18beab22152 in 434ms, sequenceid=9, compaction requested=false 2023-02-13 00:54:33,874 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 893fe9ccf4b1790a755cd18beab22152: 2023-02-13 00:54:33,874 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:33,874 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:33,874 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:33,874 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,874 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 00:54:33,874 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 00:54:33,877 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,877 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:33,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:33,877 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,877 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:33,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:33,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:33,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:33,879 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 00:54:33,879 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1cd8e999[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:33,879 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:33,879 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,887 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,888 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:33,888 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 00:54:33,888 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:33,888 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,888 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:33,889 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,889 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 00:54:33,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:33,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:33,889 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:33,890 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 00:54:33,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:33,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:33,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:33,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:33,892 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:33,892 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 00:54:33,892 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:33,892 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:33,892 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,892 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:33,895 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,895 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:33,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:33,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,895 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:33,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:33,896 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:33,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:33,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:33,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,897 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:33,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,900 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:33,900 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:33,900 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:33,900 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:33,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:33,900 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 00:54:33,900 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:33,900 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,900 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,900 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:33,900 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,900 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,900 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:33,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:33,900 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:33,900 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:33,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:34,001 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:34,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:34,006 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:34,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:34,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,008 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 00:54:34,008 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:34,008 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 00:54:34,008 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:34,009 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,009 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,010 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:34,010 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:34,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:34,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:34,010 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:34,011 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:34,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,011 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:34,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,011 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,012 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 00:54:34,012 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,012 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 00:54:34,012 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:34,012 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 00:54:34,012 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:34,012 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:34,012 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:34,012 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. started... 2023-02-13 00:54:34,012 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 893fe9ccf4b1790a755cd18beab22152 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:34,427 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512013b3ebaba21d446228147c58ba129b5b0 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512013b3ebaba21d446228147c58ba129b5b0 2023-02-13 00:54:34,834 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/.tmp/family1/98f69eac2d6043139ba1ba5c6e675c50 2023-02-13 00:54:34,840 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/.tmp/family1/98f69eac2d6043139ba1ba5c6e675c50 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/98f69eac2d6043139ba1ba5c6e675c50 2023-02-13 00:54:34,844 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/98f69eac2d6043139ba1ba5c6e675c50, entries=1, sequenceid=13, filesize=4.9 K 2023-02-13 00:54:34,845 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 893fe9ccf4b1790a755cd18beab22152 in 833ms, sequenceid=13, compaction requested=true 2023-02-13 00:54:34,845 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 893fe9ccf4b1790a755cd18beab22152: 2023-02-13 00:54:34,846 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:34,846 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:34,846 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:34,846 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:34,846 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 00:54:34,846 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 00:54:34,847 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,847 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:34,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:34,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:34,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:34,848 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,848 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:34,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:34,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:34,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:34,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:34,849 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 00:54:34,849 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1f9104f5[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:34,849 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:34,849 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,851 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,851 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:34,851 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 00:54:34,851 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:34,851 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:34,851 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:34,853 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 00:54:34,853 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:34,853 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:34,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:34,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:34,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:34,853 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 00:54:34,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:34,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:34,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:34,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:34,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:34,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:34,855 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:34,855 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 00:54:34,855 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:34,855 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:34,855 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,855 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:34,857 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,857 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:34,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:34,857 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:34,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:34,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:34,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:34,857 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:34,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:34,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,858 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:34,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:34,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:34,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:34,860 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:34,861 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:34,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:34,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:34,861 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:34,861 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:34,861 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 00:54:34,861 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:34,861 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,861 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 00:54:34,861 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:34,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:34,861 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:34,861 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,861 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:34,861 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:34,861 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,861 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,961 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:34,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:34,967 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:34,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:34,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,969 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 00:54:34,969 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:34,969 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 00:54:34,969 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:34,969 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,969 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,970 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:34,970 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:34,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:34,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:34,971 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:34,971 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:34,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,971 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:34,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,971 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,972 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 00:54:34,972 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:34,972 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 00:54:34,972 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:34,972 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 00:54:34,972 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:34,972 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:34,972 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:34,972 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. started... 2023-02-13 00:54:34,972 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 893fe9ccf4b1790a755cd18beab22152 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:35,388 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120570871c5d63cb476cb8a3450834a93c83 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120570871c5d63cb476cb8a3450834a93c83 2023-02-13 00:54:35,794 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/.tmp/family1/891fd140df60492d9d32dd1e53b922d5 2023-02-13 00:54:35,800 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/.tmp/family1/891fd140df60492d9d32dd1e53b922d5 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/891fd140df60492d9d32dd1e53b922d5 2023-02-13 00:54:35,805 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/891fd140df60492d9d32dd1e53b922d5, entries=1, sequenceid=17, filesize=4.9 K 2023-02-13 00:54:35,806 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 893fe9ccf4b1790a755cd18beab22152 in 834ms, sequenceid=17, compaction requested=true 2023-02-13 00:54:35,806 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 893fe9ccf4b1790a755cd18beab22152: 2023-02-13 00:54:35,806 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:35,806 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:35,806 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:35,806 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:35,807 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 00:54:35,807 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 00:54:35,809 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:35,809 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:35,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:35,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:35,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:35,810 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,810 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:35,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:35,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:35,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:35,814 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 00:54:35,814 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6b8ecb1d[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:35,814 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:35,814 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,816 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,816 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:35,816 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 00:54:35,816 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:35,816 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:35,816 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:35,818 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 00:54:35,818 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:35,818 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:35,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:35,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:35,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:35,818 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 00:54:35,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:35,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:35,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:35,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:35,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:35,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:35,825 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:35,825 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 00:54:35,825 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:35,825 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:35,825 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,825 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:35,827 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,827 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:35,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:35,827 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:35,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,828 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:35,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:35,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:35,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:35,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:35,829 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:35,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:35,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:35,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:35,832 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:35,832 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:35,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:35,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:35,832 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,832 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,832 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:35,832 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,832 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,832 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,832 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:35,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:35,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:35,833 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 00:54:35,836 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:35,836 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 00:54:35,836 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:35,836 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:35,836 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:35,937 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:35,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:35,942 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:35,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:35,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,944 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 00:54:35,944 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:35,944 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 00:54:35,944 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:35,944 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,945 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,946 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:35,946 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:35,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:35,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:35,946 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:35,946 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:35,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,947 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:35,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,947 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,948 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 00:54:35,948 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:35,948 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 00:54:35,948 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:35,948 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 00:54:35,948 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:35,948 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:35,948 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:35,948 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. started... 2023-02-13 00:54:35,948 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 893fe9ccf4b1790a755cd18beab22152 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:36,366 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512285aab5da43c9940d0ad66b853ce03a1f0 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512285aab5da43c9940d0ad66b853ce03a1f0 2023-02-13 00:54:36,378 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/.tmp/family1/e9eb471d56844691b567035abaa201fc 2023-02-13 00:54:36,383 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/.tmp/family1/e9eb471d56844691b567035abaa201fc as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/e9eb471d56844691b567035abaa201fc 2023-02-13 00:54:36,388 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/e9eb471d56844691b567035abaa201fc, entries=1, sequenceid=21, filesize=4.9 K 2023-02-13 00:54:36,389 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 893fe9ccf4b1790a755cd18beab22152 in 441ms, sequenceid=21, compaction requested=true 2023-02-13 00:54:36,389 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 893fe9ccf4b1790a755cd18beab22152: 2023-02-13 00:54:36,389 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:36,389 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:36,389 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:36,389 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:36,389 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 00:54:36,389 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 00:54:36,394 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,394 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:36,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:36,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:36,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:36,395 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,395 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:36,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:36,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:36,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:36,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:36,396 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 00:54:36,396 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1da31ef[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:36,396 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:36,396 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,397 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,398 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:36,398 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 00:54:36,398 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:36,398 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:36,398 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:36,400 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 00:54:36,400 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:36,400 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:36,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:36,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:36,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:36,400 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 00:54:36,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:36,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:36,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:36,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:36,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:36,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:36,403 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:36,403 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 00:54:36,403 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:36,403 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:36,403 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,403 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:36,404 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,404 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:36,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,404 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:36,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:36,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:36,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:36,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:36,405 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:36,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:36,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,406 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:36,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:36,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:36,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:36,409 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:36,409 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,409 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:36,409 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,409 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:36,409 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 00:54:36,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 00:54:36,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:36,409 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:36,409 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:36,409 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:36,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:36,409 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,409 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,409 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:36,409 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:36,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:36,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:36,510 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:36,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:36,515 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:36,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:36,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,516 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 00:54:36,516 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:36,516 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 00:54:36,516 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:36,517 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,517 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,518 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:36,518 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:36,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:36,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:36,518 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:36,518 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:36,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,519 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:36,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,519 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,519 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 00:54:36,519 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:36,519 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 00:54:36,519 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:36,519 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 00:54:36,520 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:36,520 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:36,520 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:36,520 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. started... 2023-02-13 00:54:36,520 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 893fe9ccf4b1790a755cd18beab22152 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:36,932 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512315aae0bf9b03a4e708c7c55c1fea2d081 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512315aae0bf9b03a4e708c7c55c1fea2d081 2023-02-13 00:54:37,338 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/.tmp/family1/aac231bcea454f13ab6d5c3bdd560532 2023-02-13 00:54:37,344 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/.tmp/family1/aac231bcea454f13ab6d5c3bdd560532 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/aac231bcea454f13ab6d5c3bdd560532 2023-02-13 00:54:37,348 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/aac231bcea454f13ab6d5c3bdd560532, entries=1, sequenceid=25, filesize=4.9 K 2023-02-13 00:54:37,349 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 893fe9ccf4b1790a755cd18beab22152 in 829ms, sequenceid=25, compaction requested=true 2023-02-13 00:54:37,349 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 893fe9ccf4b1790a755cd18beab22152: 2023-02-13 00:54:37,349 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:37,349 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:37,349 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:37,349 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:37,349 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 00:54:37,350 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 00:54:37,351 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:37,351 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:37,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:37,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:37,352 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,352 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:37,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:37,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:37,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:37,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:37,354 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 00:54:37,354 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a15846c[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:37,354 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:37,354 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,355 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,355 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:37,355 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:37,355 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:37,355 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 00:54:37,356 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:37,357 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 00:54:37,357 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:37,357 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:37,357 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 00:54:37,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:37,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:37,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:37,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:37,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:37,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:37,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:37,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:37,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:37,360 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:37,360 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 00:54:37,360 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:37,360 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:37,360 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,360 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:37,362 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 00:54:37,362 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,362 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:37,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:37,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,363 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:37,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:37,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:37,363 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:37,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:37,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:37,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,369 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:37,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:37,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:37,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:37,378 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:37,378 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:37,378 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,378 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 00:54:37,378 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 00:54:37,378 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:37,378 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,379 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:37,379 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,379 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,379 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:37,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:37,378 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:37,378 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:37,379 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:37,379 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:37,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:37,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:37,479 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:37,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:37,484 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:37,486 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:37,486 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,486 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 00:54:37,486 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:37,486 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 00:54:37,486 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:37,486 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,487 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,488 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:37,488 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:37,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:37,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:37,488 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:37,488 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:37,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,488 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:37,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,489 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,489 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 00:54:37,489 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:37,489 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 00:54:37,489 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:37,489 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 00:54:37,490 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:37,490 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:37,490 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:37,490 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. started... 2023-02-13 00:54:37,490 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 893fe9ccf4b1790a755cd18beab22152 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:37,904 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101e31cd287b1d147a9804abf37a9882d10 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101e31cd287b1d147a9804abf37a9882d10 2023-02-13 00:54:38,310 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/.tmp/family1/09f63698ed694e26b404e882864be069 2023-02-13 00:54:38,316 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/.tmp/family1/09f63698ed694e26b404e882864be069 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/09f63698ed694e26b404e882864be069 2023-02-13 00:54:38,320 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/09f63698ed694e26b404e882864be069, entries=1, sequenceid=29, filesize=4.9 K 2023-02-13 00:54:38,321 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 893fe9ccf4b1790a755cd18beab22152 in 831ms, sequenceid=29, compaction requested=true 2023-02-13 00:54:38,321 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 893fe9ccf4b1790a755cd18beab22152: 2023-02-13 00:54:38,321 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:38,321 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:38,321 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:38,321 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:38,321 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 00:54:38,321 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 00:54:38,322 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,322 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:38,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:38,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:38,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:38,323 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,323 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:38,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:38,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:38,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:38,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:38,324 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 00:54:38,324 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4d349577[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:38,324 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:38,324 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,326 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,327 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:38,327 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 00:54:38,327 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:38,327 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:38,327 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:38,328 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:38,328 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 00:54:38,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:38,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:38,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:38,328 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:38,328 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 00:54:38,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:38,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:38,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:38,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:38,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:38,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:38,330 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:38,330 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 00:54:38,330 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:38,330 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:38,330 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,331 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:38,332 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,332 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,332 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:38,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:38,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:38,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:38,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:38,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:38,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,333 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:38,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:38,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:38,334 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:38,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:38,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:38,336 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:38,336 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:38,336 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:38,336 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 00:54:38,336 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:38,336 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:38,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:38,336 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:38,336 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,336 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:38,336 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,336 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,336 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,336 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:38,337 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:38,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:38,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:38,437 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:38,438 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:38,441 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:38,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:38,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,443 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 00:54:38,443 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:38,443 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 00:54:38,443 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:38,443 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,443 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,448 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:38,448 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:38,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:38,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:38,448 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:38,448 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:38,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,449 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:38,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,449 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,449 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 00:54:38,449 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:38,449 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 00:54:38,449 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:38,450 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 00:54:38,450 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:38,450 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:38,450 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:38,450 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. started... 2023-02-13 00:54:38,450 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 893fe9ccf4b1790a755cd18beab22152 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:38,863 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103c71e8d35b62d445ea37a8dc9016fd716 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103c71e8d35b62d445ea37a8dc9016fd716 2023-02-13 00:54:39,269 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/.tmp/family1/be5f6f831f304e61b6bf2ae2f314b6fe 2023-02-13 00:54:39,278 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/.tmp/family1/be5f6f831f304e61b6bf2ae2f314b6fe as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/be5f6f831f304e61b6bf2ae2f314b6fe 2023-02-13 00:54:39,283 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/be5f6f831f304e61b6bf2ae2f314b6fe, entries=1, sequenceid=33, filesize=4.9 K 2023-02-13 00:54:39,284 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 893fe9ccf4b1790a755cd18beab22152 in 834ms, sequenceid=33, compaction requested=true 2023-02-13 00:54:39,284 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 893fe9ccf4b1790a755cd18beab22152: 2023-02-13 00:54:39,285 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:39,285 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:39,285 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:39,285 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:39,285 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 00:54:39,285 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 00:54:39,286 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,286 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:39,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:39,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:39,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:39,287 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,287 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:39,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:39,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:39,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:39,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:39,288 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 00:54:39,289 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:39,289 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@50ff79ef[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:39,289 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,290 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,290 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:39,290 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 00:54:39,290 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:39,290 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:39,290 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:39,292 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:39,292 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 00:54:39,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:39,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:39,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:39,292 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:39,292 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 00:54:39,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:39,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:39,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:39,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:39,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:39,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:39,294 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:39,294 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 00:54:39,295 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:39,295 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:39,295 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,295 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:39,297 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,297 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:39,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:39,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,297 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:39,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:39,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:39,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:39,298 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:39,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:39,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,299 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:39,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:39,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:39,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:39,309 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:39,309 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,309 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:39,309 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:39,309 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:39,309 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 00:54:39,309 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:39,309 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 00:54:39,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:39,309 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:39,309 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:39,309 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,309 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,309 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,309 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:39,310 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:39,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:39,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:39,410 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:39,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:39,414 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:39,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:39,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,416 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 00:54:39,416 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:39,416 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 00:54:39,416 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:39,417 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,417 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,418 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:39,418 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:39,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:39,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:39,418 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:39,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,418 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:39,419 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:39,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,419 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,420 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 00:54:39,420 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:39,420 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 00:54:39,420 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:39,420 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 00:54:39,420 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:39,420 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:39,420 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:39,420 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. started... 2023-02-13 00:54:39,420 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 893fe9ccf4b1790a755cd18beab22152 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:39,846 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907d71400a7ee1545cb843f3ad4b1eb6d54 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20150907d71400a7ee1545cb843f3ad4b1eb6d54 2023-02-13 00:54:40,263 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/.tmp/family1/7a87fc7ba5b449a5b8de07fb6e1d53d8 2023-02-13 00:54:40,269 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/.tmp/family1/7a87fc7ba5b449a5b8de07fb6e1d53d8 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/7a87fc7ba5b449a5b8de07fb6e1d53d8 2023-02-13 00:54:40,273 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/7a87fc7ba5b449a5b8de07fb6e1d53d8, entries=1, sequenceid=37, filesize=4.9 K 2023-02-13 00:54:40,274 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 893fe9ccf4b1790a755cd18beab22152 in 854ms, sequenceid=37, compaction requested=true 2023-02-13 00:54:40,274 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 893fe9ccf4b1790a755cd18beab22152: 2023-02-13 00:54:40,274 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:40,274 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:40,274 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:40,274 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:40,274 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 00:54:40,274 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 00:54:40,276 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:40,276 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:40,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:40,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:40,276 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,276 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:40,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:40,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:40,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:40,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:40,278 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 00:54:40,278 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7a7f56[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:40,278 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:40,278 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,280 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,280 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:40,280 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 00:54:40,280 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:40,281 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:40,281 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:40,282 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:40,282 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 00:54:40,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:40,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:40,282 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:40,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:40,282 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 00:54:40,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:40,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:40,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:40,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:40,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:40,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:40,285 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:40,285 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 00:54:40,285 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:40,285 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:40,285 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,285 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:40,287 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,287 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:40,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:40,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,287 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:40,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,287 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:40,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:40,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:40,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:40,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:40,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,288 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:40,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:40,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:40,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:40,290 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:40,290 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,290 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:40,290 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,291 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:40,291 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 00:54:40,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:40,291 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 00:54:40,291 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:40,291 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:40,291 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:40,291 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:40,291 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,291 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,291 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:40,291 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:40,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:40,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:40,391 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:40,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:40,397 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:40,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:40,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,399 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 00:54:40,399 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:40,400 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 00:54:40,400 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 00:54:40,400 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,400 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,403 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:40,403 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:40,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:40,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:40,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,403 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:40,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,404 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,404 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 00:54:40,404 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:40,404 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 00:54:40,404 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:40,404 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 00:54:40,405 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:40,405 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:40,405 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:40,405 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:40,405 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:40,405 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. started... 2023-02-13 00:54:40,405 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 893fe9ccf4b1790a755cd18beab22152 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:40,824 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112028b9e52c5f0e45dcb51b5279210b376b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015112028b9e52c5f0e45dcb51b5279210b376b 2023-02-13 00:54:41,230 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/.tmp/family1/aec17f04e9d94ff9bd1fb075c2746cc2 2023-02-13 00:54:41,236 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/.tmp/family1/aec17f04e9d94ff9bd1fb075c2746cc2 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/aec17f04e9d94ff9bd1fb075c2746cc2 2023-02-13 00:54:41,241 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/aec17f04e9d94ff9bd1fb075c2746cc2, entries=1, sequenceid=41, filesize=4.9 K 2023-02-13 00:54:41,242 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 893fe9ccf4b1790a755cd18beab22152 in 837ms, sequenceid=41, compaction requested=true 2023-02-13 00:54:41,242 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 893fe9ccf4b1790a755cd18beab22152: 2023-02-13 00:54:41,242 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:41,242 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:41,242 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:41,242 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:41,242 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 00:54:41,242 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 00:54:41,243 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:41,243 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:41,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:41,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:41,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:41,244 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:41,244 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:41,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:41,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:41,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:41,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:41,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:41,245 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 00:54:41,245 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76baadd1[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:41,245 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 00:54:41,245 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:41,246 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:41,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:41,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:41,247 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 00:54:41,247 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:41,247 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 00:54:41,247 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:41,247 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:41,249 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:41,249 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 00:54:41,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:41,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:41,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:41,249 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:41,249 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 00:54:41,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:41,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:41,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:41,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:41,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:41,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:41,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:41,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:41,252 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:54:41,252 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 00:54:41,252 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:41,252 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:41,252 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:41,252 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:41,254 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:41,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:41,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:41,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:41,255 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:41,255 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:41,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:41,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:41,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:41,255 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:41,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:41,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:41,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:41,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:41,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:41,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:41,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:41,256 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:41,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:41,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:41,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:41,261 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:41,261 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:41,261 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:41,261 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:41,261 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:41,261 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 00:54:41,261 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:41,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:41,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:41,261 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:41,261 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:41,262 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:41,262 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:41,262 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:41,262 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:41,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:41,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:41,262 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:41,362 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:41,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:41,364 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43563] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-02-13 00:54:41,366 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] procedure2.ProcedureExecutor(1029): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 00:54:41,367 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 00:54:41,367 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:54:41,367 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676250281367 2023-02-13 00:54:41,369 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 00:54:41,370 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-02-13 00:54:41,370 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 00:54:41,370 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] zookeeper.ReadOnlyZKClient(139): Connect 0x7bfba985 to 127.0.0.1:54478 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:54:41,374 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4588097a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:54:41,374 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 00:54:42,213 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103feec7d6d955c43e8bdaad4c25c3896f5 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103feec7d6d955c43e8bdaad4c25c3896f5 2023-02-13 00:54:42,215 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 00:54:42,229 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:54:42,230 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59726, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:54:42,238 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 00:54:42,239 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40192, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 00:54:42,248 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3b2e0be0ad744c58a2f7238c015ca205 first=Optional[r3] last=Optional[r6] 2023-02-13 00:54:42,249 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3b2e0be0ad744c58a2f7238c015ca205}] 2023-02-13 00:54:42,255 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3b2e0be0ad744c58a2f7238c015ca205 for inclusion in 893fe9ccf4b1790a755cd18beab22152/family1 2023-02-13 00:54:42,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-13 00:54:42,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 00:54:42,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for 893fe9ccf4b1790a755cd18beab22152: 2023-02-13 00:54:42,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3b2e0be0ad744c58a2f7238c015ca205 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyPolicy__rsb4lut4bflm1h34npr2jvah6vdsk1mmjjh95tcl8vnoiile3q40eqglcrkgk01c/family1/3b2e0be0ad744c58a2f7238c015ca205 2023-02-13 00:54:42,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyPolicy__rsb4lut4bflm1h34npr2jvah6vdsk1mmjjh95tcl8vnoiile3q40eqglcrkgk01c/family1/3b2e0be0ad744c58a2f7238c015ca205 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/860d97c9f8b844a7b04804887d6bc7d3_SeqId_44_ 2023-02-13 00:54:42,262 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyPolicy__rsb4lut4bflm1h34npr2jvah6vdsk1mmjjh95tcl8vnoiile3q40eqglcrkgk01c/family1/3b2e0be0ad744c58a2f7238c015ca205 into 893fe9ccf4b1790a755cd18beab22152/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/860d97c9f8b844a7b04804887d6bc7d3_SeqId_44_ - updating store file list. 2023-02-13 00:54:42,266 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/860d97c9f8b844a7b04804887d6bc7d3_SeqId_44_ into 893fe9ccf4b1790a755cd18beab22152/family1 2023-02-13 00:54:42,266 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyPolicy__rsb4lut4bflm1h34npr2jvah6vdsk1mmjjh95tcl8vnoiile3q40eqglcrkgk01c/family1/3b2e0be0ad744c58a2f7238c015ca205 into 893fe9ccf4b1790a755cd18beab22152/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/860d97c9f8b844a7b04804887d6bc7d3_SeqId_44_) 2023-02-13 00:54:42,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyPolicy__rsb4lut4bflm1h34npr2jvah6vdsk1mmjjh95tcl8vnoiile3q40eqglcrkgk01c/family1/3b2e0be0ad744c58a2f7238c015ca205 2023-02-13 00:54:42,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-13 00:54:42,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-13 00:54:42,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(1912): 893fe9ccf4b1790a755cd18beab22152/family1 is initiating minor compaction 2023-02-13 00:54:42,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5d0f7ed0; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:42,270 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 893fe9ccf4b1790a755cd18beab22152/family1 in testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:42,270 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/2aa13bbd06934c8f97c366f4c7a817d2, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/98f69eac2d6043139ba1ba5c6e675c50, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/891fd140df60492d9d32dd1e53b922d5, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/e9eb471d56844691b567035abaa201fc, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/aac231bcea454f13ab6d5c3bdd560532, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/09f63698ed694e26b404e882864be069, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/be5f6f831f304e61b6bf2ae2f314b6fe, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/7a87fc7ba5b449a5b8de07fb6e1d53d8, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/aec17f04e9d94ff9bd1fb075c2746cc2, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/860d97c9f8b844a7b04804887d6bc7d3_SeqId_44_] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/.tmp, totalSize=50.0 K 2023-02-13 00:54:42,270 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 2aa13bbd06934c8f97c366f4c7a817d2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-13 00:54:42,271 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 98f69eac2d6043139ba1ba5c6e675c50, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-13 00:54:42,271 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 891fd140df60492d9d32dd1e53b922d5, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-13 00:54:42,271 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting e9eb471d56844691b567035abaa201fc, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-13 00:54:42,271 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting aac231bcea454f13ab6d5c3bdd560532, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-13 00:54:42,271 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 09f63698ed694e26b404e882864be069, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-13 00:54:42,271 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting be5f6f831f304e61b6bf2ae2f314b6fe, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-13 00:54:42,271 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 7a87fc7ba5b449a5b8de07fb6e1d53d8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-13 00:54:42,271 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting aec17f04e9d94ff9bd1fb075c2746cc2, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-13 00:54:42,271 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 860d97c9f8b844a7b04804887d6bc7d3_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-13 00:54:42,272 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 00:54:42,273 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:42,276 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512285aab5da43c9940d0ad66b853ce03a1f0 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512285aab5da43c9940d0ad66b853ce03a1f0 2023-02-13 00:54:42,277 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512315aae0bf9b03a4e708c7c55c1fea2d081 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512315aae0bf9b03a4e708c7c55c1fea2d081 2023-02-13 00:54:42,278 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101e31cd287b1d147a9804abf37a9882d10 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101e31cd287b1d147a9804abf37a9882d10 2023-02-13 00:54:42,280 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103c71e8d35b62d445ea37a8dc9016fd716 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103c71e8d35b62d445ea37a8dc9016fd716 2023-02-13 00:54:42,280 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-02-13 00:54:42,280 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-13 00:54:42,280 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 00:54:42,280 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 00:54:42,281 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 00:54:42,281 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 00:54:42,281 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 00:54:42,281 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 00:54:42,281 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 00:54:42,281 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 00:54:42,318 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] throttle.PressureAwareThroughputController(145): 893fe9ccf4b1790a755cd18beab22152#family1#compaction#48 average throughput is 0.45 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:54:42,347 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/.tmp/family1/531f870fc7fe4c56b02dd30ff43c215b as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/531f870fc7fe4c56b02dd30ff43c215b 2023-02-13 00:54:42,352 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205d495e71805ad4a49b60dfc3f2fb37d2c to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205d495e71805ad4a49b60dfc3f2fb37d2c 2023-02-13 00:54:42,353 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 00:54:42,365 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 893fe9ccf4b1790a755cd18beab22152/family1 of 893fe9ccf4b1790a755cd18beab22152 into 531f870fc7fe4c56b02dd30ff43c215b(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:54:42,366 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 893fe9ccf4b1790a755cd18beab22152: 2023-02-13 00:54:42,366 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1676249682270; duration=0sec 2023-02-13 00:54:42,366 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:42,381 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b97fa64bdcd249b4928d63a0b01b3dc6 first=Optional[r00] last=Optional[r2] 2023-02-13 00:54:42,382 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b97fa64bdcd249b4928d63a0b01b3dc6}] 2023-02-13 00:54:42,385 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b97fa64bdcd249b4928d63a0b01b3dc6 for inclusion in 893fe9ccf4b1790a755cd18beab22152/family1 2023-02-13 00:54:42,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-13 00:54:42,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 00:54:42,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for 893fe9ccf4b1790a755cd18beab22152: 2023-02-13 00:54:42,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b97fa64bdcd249b4928d63a0b01b3dc6 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyPolicy__dvj11jm0do7h73jl78a4b28jnj005aednuisv190csat7c01pc2osoov2t2sdpvr/family1/b97fa64bdcd249b4928d63a0b01b3dc6 2023-02-13 00:54:42,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyPolicy__dvj11jm0do7h73jl78a4b28jnj005aednuisv190csat7c01pc2osoov2t2sdpvr/family1/b97fa64bdcd249b4928d63a0b01b3dc6 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/8c6a44f6999e460e8449f4856d4688c7_SeqId_47_ 2023-02-13 00:54:42,398 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyPolicy__dvj11jm0do7h73jl78a4b28jnj005aednuisv190csat7c01pc2osoov2t2sdpvr/family1/b97fa64bdcd249b4928d63a0b01b3dc6 into 893fe9ccf4b1790a755cd18beab22152/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/8c6a44f6999e460e8449f4856d4688c7_SeqId_47_ - updating store file list. 2023-02-13 00:54:42,403 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/8c6a44f6999e460e8449f4856d4688c7_SeqId_47_ into 893fe9ccf4b1790a755cd18beab22152/family1 2023-02-13 00:54:42,403 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyPolicy__dvj11jm0do7h73jl78a4b28jnj005aednuisv190csat7c01pc2osoov2t2sdpvr/family1/b97fa64bdcd249b4928d63a0b01b3dc6 into 893fe9ccf4b1790a755cd18beab22152/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/8c6a44f6999e460e8449f4856d4688c7_SeqId_47_) 2023-02-13 00:54:42,403 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyPolicy__dvj11jm0do7h73jl78a4b28jnj005aednuisv190csat7c01pc2osoov2t2sdpvr/family1/b97fa64bdcd249b4928d63a0b01b3dc6 2023-02-13 00:54:42,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-13 00:54:42,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 345120 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-13 00:54:42,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(1912): 893fe9ccf4b1790a755cd18beab22152/family1 is initiating minor compaction (all files) 2023-02-13 00:54:42,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@346501f3; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:42,405 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 893fe9ccf4b1790a755cd18beab22152/family1 in testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:54:42,405 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/7cea6b39c5d54eed870857fbacd06638, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/531f870fc7fe4c56b02dd30ff43c215b, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/8c6a44f6999e460e8449f4856d4688c7_SeqId_47_] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/.tmp, totalSize=337.0 K 2023-02-13 00:54:42,406 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 7cea6b39c5d54eed870857fbacd06638, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878845884 2023-02-13 00:54:42,406 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 531f870fc7fe4c56b02dd30ff43c215b, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621245884 2023-02-13 00:54:42,407 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 8c6a44f6999e460e8449f4856d4688c7_SeqId_47_, keycount=1002, bloomtype=ROW, size=165.4 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878845884 2023-02-13 00:54:42,407 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 00:54:42,408 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:42,410 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015113070b56723b4b6446d85379f61d8600cdc to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015113070b56723b4b6446d85379f61d8600cdc 2023-02-13 00:54:42,411 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512013b3ebaba21d446228147c58ba129b5b0 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512013b3ebaba21d446228147c58ba129b5b0 2023-02-13 00:54:42,413 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120570871c5d63cb476cb8a3450834a93c83 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120570871c5d63cb476cb8a3450834a93c83 2023-02-13 00:54:42,413 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-02-13 00:54:42,413 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-13 00:54:42,414 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 00:54:42,414 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 00:54:42,414 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:54:42,415 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 00:54:42,417 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.0500 sec 2023-02-13 00:54:42,427 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] throttle.PressureAwareThroughputController(145): 893fe9ccf4b1790a755cd18beab22152#family1#compaction#49 average throughput is 17.18 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:54:42,458 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/2aa13bbd06934c8f97c366f4c7a817d2, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/98f69eac2d6043139ba1ba5c6e675c50, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/891fd140df60492d9d32dd1e53b922d5, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/e9eb471d56844691b567035abaa201fc, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/aac231bcea454f13ab6d5c3bdd560532, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/09f63698ed694e26b404e882864be069, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/be5f6f831f304e61b6bf2ae2f314b6fe, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/7a87fc7ba5b449a5b8de07fb6e1d53d8, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/aec17f04e9d94ff9bd1fb075c2746cc2, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/860d97c9f8b844a7b04804887d6bc7d3_SeqId_44_] to archive 2023-02-13 00:54:42,459 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:42,463 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/2aa13bbd06934c8f97c366f4c7a817d2 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/2aa13bbd06934c8f97c366f4c7a817d2 2023-02-13 00:54:42,465 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/98f69eac2d6043139ba1ba5c6e675c50 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/98f69eac2d6043139ba1ba5c6e675c50 2023-02-13 00:54:42,472 DEBUG [hfile_cleaner-dir-scan-pool-0] cleaner.CleanerChore(483): Couldn't delete 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyPolicy' yet because it isn't empty w/exception. org.apache.hadoop.fs.PathIsNotEmptyDirectoryException: ``/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyPolicy is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) ': Directory is not empty: `/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyPolicy is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1595) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:881) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:878) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:878) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$4(CleanerChore.java:453) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:478) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:453) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:792) at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2153) at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:61) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:439) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$2(CleanerChore.java:432) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): `/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyPolicy is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.delete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:572) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1593) ... 29 more 2023-02-13 00:54:42,474 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/891fd140df60492d9d32dd1e53b922d5 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/891fd140df60492d9d32dd1e53b922d5 2023-02-13 00:54:42,476 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/e9eb471d56844691b567035abaa201fc to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/e9eb471d56844691b567035abaa201fc 2023-02-13 00:54:42,478 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/aac231bcea454f13ab6d5c3bdd560532 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/aac231bcea454f13ab6d5c3bdd560532 2023-02-13 00:54:42,480 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/09f63698ed694e26b404e882864be069 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/09f63698ed694e26b404e882864be069 2023-02-13 00:54:42,482 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/be5f6f831f304e61b6bf2ae2f314b6fe to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/be5f6f831f304e61b6bf2ae2f314b6fe 2023-02-13 00:54:42,486 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/7a87fc7ba5b449a5b8de07fb6e1d53d8 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/7a87fc7ba5b449a5b8de07fb6e1d53d8 2023-02-13 00:54:42,488 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/aec17f04e9d94ff9bd1fb075c2746cc2 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/aec17f04e9d94ff9bd1fb075c2746cc2 2023-02-13 00:54:42,490 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/860d97c9f8b844a7b04804887d6bc7d3_SeqId_44_ to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/860d97c9f8b844a7b04804887d6bc7d3_SeqId_44_ 2023-02-13 00:54:42,554 INFO [Listener at localhost/37869] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=260 (was 247) - Thread LEAK? -, OpenFileDescriptor=629 (was 590) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=216 (was 214) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3819 (was 4319) 2023-02-13 00:54:42,566 INFO [Listener at localhost/37869] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=260, OpenFileDescriptor=629, MaxFileDescriptor=60000, SystemLoadAverage=216, ProcessCount=168, AvailableMemoryMB=3818 2023-02-13 00:54:42,568 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-13 00:54:42,569 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure2.ProcedureExecutor(1029): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:42,571 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 00:54:42,571 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-02-13 00:54:42,571 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 00:54:42,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-13 00:54:42,573 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:42,573 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3 empty. 2023-02-13 00:54:42,574 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:42,574 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-02-13 00:54:42,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-13 00:54:42,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-13 00:54:42,983 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-02-13 00:54:42,985 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7671): creating {ENCODED => e740718b17018a47b245263e5bd100e3, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp 2023-02-13 00:54:42,991 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:42,991 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1603): Closing e740718b17018a47b245263e5bd100e3, disabling compactions & flushes 2023-02-13 00:54:42,992 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:42,992 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:42,992 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. after waiting 0 ms 2023-02-13 00:54:42,992 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:42,992 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:42,992 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1557): Region close journal for e740718b17018a47b245263e5bd100e3: Waiting for close lock at 1676249682991Disabling compacts and flushes for region at 1676249682991Disabling writes for close at 1676249682992 (+1 ms)Writing region close event to WAL at 1676249682992Closed at 1676249682992 2023-02-13 00:54:42,994 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 00:54:42,996 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676249682995"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249682995"}]},"ts":"1676249682995"} 2023-02-13 00:54:42,997 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 00:54:42,998 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 00:54:42,998 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249682998"}]},"ts":"1676249682998"} 2023-02-13 00:54:43,000 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-02-13 00:54:43,003 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=e740718b17018a47b245263e5bd100e3, ASSIGN}] 2023-02-13 00:54:43,005 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=e740718b17018a47b245263e5bd100e3, ASSIGN 2023-02-13 00:54:43,005 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=e740718b17018a47b245263e5bd100e3, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=false 2023-02-13 00:54:43,156 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=e740718b17018a47b245263e5bd100e3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:43,157 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676249683156"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249683156"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249683156"}]},"ts":"1676249683156"} 2023-02-13 00:54:43,158 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure e740718b17018a47b245263e5bd100e3, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:54:43,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-13 00:54:43,261 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/.tmp/family1/b82a4a6e3a3a429c941a7455432ba8b8 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/b82a4a6e3a3a429c941a7455432ba8b8 2023-02-13 00:54:43,268 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 893fe9ccf4b1790a755cd18beab22152/family1 of 893fe9ccf4b1790a755cd18beab22152 into b82a4a6e3a3a429c941a7455432ba8b8(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:54:43,268 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 893fe9ccf4b1790a755cd18beab22152: 2023-02-13 00:54:43,268 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152., storeName=family1, fileCount=3, fileSize=337.0 K (165.1 K, 6.6 K, 165.4 K), priority=2, time=1676249682405; duration=0sec 2023-02-13 00:54:43,268 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:43,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:43,313 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e740718b17018a47b245263e5bd100e3, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:54:43,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:43,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:43,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:43,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:43,315 INFO [StoreOpener-e740718b17018a47b245263e5bd100e3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:43,316 DEBUG [StoreOpener-e740718b17018a47b245263e5bd100e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1 2023-02-13 00:54:43,317 DEBUG [StoreOpener-e740718b17018a47b245263e5bd100e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1 2023-02-13 00:54:43,317 INFO [StoreOpener-e740718b17018a47b245263e5bd100e3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e740718b17018a47b245263e5bd100e3 columnFamilyName family1 2023-02-13 00:54:43,318 INFO [StoreOpener-e740718b17018a47b245263e5bd100e3-1] regionserver.HStore(310): Store=e740718b17018a47b245263e5bd100e3/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:43,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:43,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:43,321 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:43,323 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:54:43,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e740718b17018a47b245263e5bd100e3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63796858, jitterRate=-0.0493527352809906}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 00:54:43,324 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e740718b17018a47b245263e5bd100e3: Running coprocessor pre-open hook at 1676249683314Writing region info on filesystem at 1676249683314Initializing all the Stores at 1676249683315 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676249683315Cleaning up temporary data from old regions at 1676249683319 (+4 ms)Cleaning up detritus from prior splits at 1676249683320 (+1 ms)Running coprocessor post-open hooks at 1676249683324 (+4 ms)Region opened successfully at 1676249683324 2023-02-13 00:54:43,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3., pid=41, masterSystemTime=1676249683310 2023-02-13 00:54:43,326 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:43,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:43,326 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=e740718b17018a47b245263e5bd100e3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:43,327 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676249683326"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249683326"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249683326"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249683326"}]},"ts":"1676249683326"} 2023-02-13 00:54:43,330 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=40 2023-02-13 00:54:43,330 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure e740718b17018a47b245263e5bd100e3, server=jenkins-hbase4.apache.org,36605,1676249622318 in 170 msec 2023-02-13 00:54:43,332 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=39 2023-02-13 00:54:43,332 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=e740718b17018a47b245263e5bd100e3, ASSIGN in 327 msec 2023-02-13 00:54:43,333 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 00:54:43,333 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249683333"}]},"ts":"1676249683333"} 2023-02-13 00:54:43,334 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-02-13 00:54:43,337 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 00:54:43,338 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 769 msec 2023-02-13 00:54:43,346 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/7cea6b39c5d54eed870857fbacd06638, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/531f870fc7fe4c56b02dd30ff43c215b, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/8c6a44f6999e460e8449f4856d4688c7_SeqId_47_] to archive 2023-02-13 00:54:43,347 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:43,349 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/7cea6b39c5d54eed870857fbacd06638 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/7cea6b39c5d54eed870857fbacd06638 2023-02-13 00:54:43,350 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/531f870fc7fe4c56b02dd30ff43c215b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/531f870fc7fe4c56b02dd30ff43c215b 2023-02-13 00:54:43,351 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/8c6a44f6999e460e8449f4856d4688c7_SeqId_47_ to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/family1/8c6a44f6999e460e8449f4856d4688c7_SeqId_47_ 2023-02-13 00:54:43,605 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 00:54:43,636 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 00:54:43,677 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-13 00:54:43,677 INFO [Listener at localhost/37869] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-02-13 00:54:43,703 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:43,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:43,708 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 00:54:43,708 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:43,708 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 00:54:43,708 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 00:54:43,708 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:43,709 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:43,710 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:43,710 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:43,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:43,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:43,710 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:43,710 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:43,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:43,711 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:43,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:43,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:43,713 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:43,713 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:43,713 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 00:54:43,713 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:43,713 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 00:54:43,713 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:43,713 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:43,714 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:43,714 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. started... 2023-02-13 00:54:43,714 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e740718b17018a47b245263e5bd100e3 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-13 00:54:44,136 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130a532a233d4b3487b8e2d4199e1e87b67 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130a532a233d4b3487b8e2d4199e1e87b67 2023-02-13 00:54:44,541 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/.tmp/family1/17a477ffebe040ccb4e5183f637fe0d1 2023-02-13 00:54:44,546 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/.tmp/family1/17a477ffebe040ccb4e5183f637fe0d1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/17a477ffebe040ccb4e5183f637fe0d1 2023-02-13 00:54:44,550 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/17a477ffebe040ccb4e5183f637fe0d1, entries=1000, sequenceid=5, filesize=191.4 K 2023-02-13 00:54:44,551 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for e740718b17018a47b245263e5bd100e3 in 836ms, sequenceid=5, compaction requested=false 2023-02-13 00:54:44,551 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e740718b17018a47b245263e5bd100e3: 2023-02-13 00:54:44,551 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:44,551 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:44,552 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:44,552 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:44,552 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 00:54:44,552 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 00:54:44,553 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:44,553 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:44,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:44,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:44,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:44,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:44,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:44,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:44,555 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 00:54:44,555 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1f4c015c[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:44,555 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 00:54:44,555 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,556 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,557 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 00:54:44,557 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 00:54:44,557 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:44,557 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:44,557 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:44,559 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 00:54:44,559 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:44,559 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:44,559 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 00:54:44,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:44,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:44,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:44,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:44,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:44,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:44,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:44,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:44,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:44,561 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 00:54:44,561 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 00:54:44,561 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:44,561 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:44,561 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,561 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:44,563 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,563 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:44,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,563 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:44,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:44,563 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:44,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,564 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:44,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:44,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:44,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:44,566 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:44,566 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,566 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:44,566 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:44,566 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:44,566 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:44,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:44,566 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 00:54:44,566 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:44,566 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-13 00:54:44,566 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:44,566 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:44,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:44,566 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,567 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,567 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:44,567 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:44,667 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:44,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:44,672 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:44,674 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:44,674 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,674 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 00:54:44,674 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:44,674 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 00:54:44,674 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 00:54:44,674 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,674 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,676 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:44,676 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:44,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:44,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:44,676 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:44,676 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:44,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,677 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:44,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,677 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,677 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 00:54:44,677 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:44,677 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 00:54:44,677 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:44,677 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 00:54:44,677 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:44,677 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:44,677 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:44,678 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. started... 2023-02-13 00:54:44,678 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e740718b17018a47b245263e5bd100e3 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-13 00:54:45,090 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128027c570cbc9248b48445dabfb4c5df52 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128027c570cbc9248b48445dabfb4c5df52 2023-02-13 00:54:45,496 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/.tmp/family1/4a7dd7f4523148e0b934b9f3e6e8dd91 2023-02-13 00:54:45,503 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/.tmp/family1/4a7dd7f4523148e0b934b9f3e6e8dd91 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/4a7dd7f4523148e0b934b9f3e6e8dd91 2023-02-13 00:54:45,509 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/4a7dd7f4523148e0b934b9f3e6e8dd91, entries=1, sequenceid=9, filesize=5.0 K 2023-02-13 00:54:45,510 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for e740718b17018a47b245263e5bd100e3 in 832ms, sequenceid=9, compaction requested=false 2023-02-13 00:54:45,510 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e740718b17018a47b245263e5bd100e3: 2023-02-13 00:54:45,510 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:45,510 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:45,511 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:45,511 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:45,511 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 00:54:45,511 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 00:54:45,513 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:45,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:45,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:45,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:45,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:45,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:45,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:45,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:45,516 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,516 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,517 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 00:54:45,517 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:45,517 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 00:54:45,517 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3457235b[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:45,517 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,518 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,519 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 00:54:45,519 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 00:54:45,519 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:45,519 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:45,519 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:45,520 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 00:54:45,520 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:45,520 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 00:54:45,521 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:45,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:45,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:45,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:45,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:45,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:45,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:45,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:45,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:45,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:45,524 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 00:54:45,524 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 00:54:45,525 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:45,525 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:45,525 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,525 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:45,528 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,528 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,528 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:45,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:45,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:45,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:45,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:45,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:45,529 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:45,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:45,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,531 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:45,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:45,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:45,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:45,538 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:45,538 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,538 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,538 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:45,538 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:45,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:45,538 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:45,538 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,538 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,538 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:45,539 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:45,539 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 00:54:45,539 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:45,539 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-13 00:54:45,539 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:45,539 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:45,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:45,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:45,639 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:45,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:45,645 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:45,658 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:45,659 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,659 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 00:54:45,659 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:45,659 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 00:54:45,659 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 00:54:45,659 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,659 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,661 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:45,661 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:45,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:45,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:45,661 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:45,661 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:45,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,663 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:45,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,663 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,670 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 00:54:45,670 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:45,670 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 00:54:45,670 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:45,671 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 00:54:45,671 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:45,671 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:45,671 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:45,671 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. started... 2023-02-13 00:54:45,671 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e740718b17018a47b245263e5bd100e3 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:46,089 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201c1ed825581454c0eaab0712eb6e80cd2 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201c1ed825581454c0eaab0712eb6e80cd2 2023-02-13 00:54:46,495 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/.tmp/family1/e6e9c8640c1a404cb5dd8912b91d3af5 2023-02-13 00:54:46,500 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/.tmp/family1/e6e9c8640c1a404cb5dd8912b91d3af5 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/e6e9c8640c1a404cb5dd8912b91d3af5 2023-02-13 00:54:46,504 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/e6e9c8640c1a404cb5dd8912b91d3af5, entries=1, sequenceid=13, filesize=5.0 K 2023-02-13 00:54:46,505 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e740718b17018a47b245263e5bd100e3 in 834ms, sequenceid=13, compaction requested=true 2023-02-13 00:54:46,505 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e740718b17018a47b245263e5bd100e3: 2023-02-13 00:54:46,505 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:46,505 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:46,505 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:46,505 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:46,505 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 00:54:46,505 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 00:54:46,508 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:46,508 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:46,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:46,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:46,508 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:46,508 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:46,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:46,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:46,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:46,509 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 00:54:46,509 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@67047a48[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:46,509 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 00:54:46,509 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,511 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,511 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 00:54:46,511 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 00:54:46,511 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:46,511 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:46,511 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:46,512 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:46,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:46,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:46,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:46,512 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 00:54:46,512 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:46,513 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 00:54:46,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:46,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:46,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:46,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:46,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:46,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:46,515 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 00:54:46,515 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 00:54:46,515 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:46,515 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:46,515 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,515 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:46,516 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,516 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,516 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:46,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:46,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:46,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:46,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:46,517 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:46,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:46,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:46,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,518 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:46,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:46,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:46,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:46,521 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:46,521 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,521 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:46,521 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,521 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:46,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:46,521 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:46,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:46,521 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 00:54:46,522 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:46,521 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:46,522 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,522 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,522 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,522 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:46,522 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:46,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:46,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:46,622 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:46,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:46,627 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:46,629 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:46,629 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,629 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 00:54:46,629 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:46,629 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 00:54:46,629 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 00:54:46,629 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,629 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,631 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:46,631 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:46,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:46,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:46,631 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:46,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,631 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:46,631 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:46,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,632 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,632 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 00:54:46,632 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:46,632 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 00:54:46,632 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:46,632 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 00:54:46,632 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:46,632 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:46,632 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:46,632 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. started... 2023-02-13 00:54:46,632 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e740718b17018a47b245263e5bd100e3 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:47,045 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120560f163a7892348b4a2c9a955fe6d3c3a to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120560f163a7892348b4a2c9a955fe6d3c3a 2023-02-13 00:54:47,451 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/.tmp/family1/af03cd173d8f495890da469d50448a4e 2023-02-13 00:54:47,456 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/.tmp/family1/af03cd173d8f495890da469d50448a4e as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/af03cd173d8f495890da469d50448a4e 2023-02-13 00:54:47,460 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/af03cd173d8f495890da469d50448a4e, entries=1, sequenceid=17, filesize=5.0 K 2023-02-13 00:54:47,460 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e740718b17018a47b245263e5bd100e3 in 828ms, sequenceid=17, compaction requested=true 2023-02-13 00:54:47,460 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e740718b17018a47b245263e5bd100e3: 2023-02-13 00:54:47,461 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:47,461 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:47,461 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:47,461 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:47,461 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 00:54:47,461 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 00:54:47,462 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:47,462 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:47,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:47,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:47,463 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,463 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:47,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:47,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:47,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:47,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:47,464 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 00:54:47,465 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@44f3875c[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:47,465 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 00:54:47,465 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,466 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,466 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 00:54:47,466 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 00:54:47,466 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:47,467 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:47,467 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:47,468 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 00:54:47,468 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:47,468 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:47,468 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 00:54:47,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:47,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:47,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:47,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:47,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:47,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:47,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:47,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:47,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:47,471 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 00:54:47,471 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 00:54:47,471 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:47,471 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:47,471 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,471 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:47,474 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,474 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,475 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:47,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:47,475 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:47,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:47,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:47,477 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:47,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:47,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:47,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,480 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:47,480 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,480 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,480 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:47,480 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,480 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:47,480 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 00:54:47,480 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:47,480 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,480 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,480 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-13 00:54:47,480 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:47,480 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:47,480 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,481 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,581 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:47,582 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:47,586 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:47,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:47,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,589 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 00:54:47,589 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:47,589 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 00:54:47,589 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 00:54:47,589 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,590 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,591 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,591 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:47,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,592 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:47,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,592 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:47,592 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:47,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,592 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,593 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 00:54:47,593 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:47,593 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 00:54:47,593 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:47,593 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 00:54:47,596 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:47,596 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:47,596 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:47,596 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. started... 2023-02-13 00:54:47,596 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e740718b17018a47b245263e5bd100e3 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:48,019 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512283a79ae107c4b46169f5d32553364f1f7 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512283a79ae107c4b46169f5d32553364f1f7 2023-02-13 00:54:48,425 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/.tmp/family1/6f035f69746f41d0964b8ca16f86cc35 2023-02-13 00:54:48,430 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/.tmp/family1/6f035f69746f41d0964b8ca16f86cc35 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/6f035f69746f41d0964b8ca16f86cc35 2023-02-13 00:54:48,434 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/6f035f69746f41d0964b8ca16f86cc35, entries=1, sequenceid=21, filesize=5.0 K 2023-02-13 00:54:48,435 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e740718b17018a47b245263e5bd100e3 in 839ms, sequenceid=21, compaction requested=true 2023-02-13 00:54:48,435 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e740718b17018a47b245263e5bd100e3: 2023-02-13 00:54:48,436 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:48,436 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:48,436 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:48,436 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:48,436 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 00:54:48,436 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 00:54:48,441 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:48,441 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:48,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:48,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:48,441 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,441 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:48,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:48,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:48,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:48,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:48,442 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 00:54:48,443 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b4a287f[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:48,443 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 00:54:48,443 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,444 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,444 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 00:54:48,444 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 00:54:48,444 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:48,444 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:48,444 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:48,445 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 00:54:48,445 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:48,445 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:48,445 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 00:54:48,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:48,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:48,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:48,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:48,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:48,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:48,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:48,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:48,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:48,448 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 00:54:48,448 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 00:54:48,448 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:48,448 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:48,448 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,448 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:48,449 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,449 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:48,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:48,449 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:48,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:48,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:48,450 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:48,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:48,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:48,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:48,451 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:48,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:48,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:48,454 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:48,454 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,454 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,454 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:48,454 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:48,454 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:48,454 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:48,454 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:48,454 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 00:54:48,455 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:48,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:48,454 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,455 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,455 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:48,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:48,455 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:48,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:48,555 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:48,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:48,560 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:48,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:48,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,561 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 00:54:48,561 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:48,561 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 00:54:48,561 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 00:54:48,562 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,562 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,563 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:48,563 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:48,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:48,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:48,563 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:48,563 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:48,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,564 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:48,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,564 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,564 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 00:54:48,565 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:48,565 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 00:54:48,565 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:48,565 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 00:54:48,565 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:48,565 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:48,565 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:48,565 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. started... 2023-02-13 00:54:48,565 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e740718b17018a47b245263e5bd100e3 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:48,847 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 00:54:48,977 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123169d57d5cb14b476882ee86e9413098a3 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015123169d57d5cb14b476882ee86e9413098a3 2023-02-13 00:54:49,383 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/.tmp/family1/4b1e45d7d59a47d8b587a179910afe45 2023-02-13 00:54:49,387 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/.tmp/family1/4b1e45d7d59a47d8b587a179910afe45 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/4b1e45d7d59a47d8b587a179910afe45 2023-02-13 00:54:49,391 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/4b1e45d7d59a47d8b587a179910afe45, entries=1, sequenceid=25, filesize=5.0 K 2023-02-13 00:54:49,392 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e740718b17018a47b245263e5bd100e3 in 827ms, sequenceid=25, compaction requested=true 2023-02-13 00:54:49,392 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e740718b17018a47b245263e5bd100e3: 2023-02-13 00:54:49,392 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:49,392 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:49,392 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:49,392 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:49,392 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 00:54:49,392 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 00:54:49,394 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:49,394 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:49,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:49,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:49,394 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,394 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:49,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:49,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:49,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:49,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:49,396 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 00:54:49,396 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@360a89b6[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:49,396 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 00:54:49,396 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,397 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,397 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 00:54:49,397 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 00:54:49,397 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:49,397 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:49,397 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:49,399 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:49,399 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 00:54:49,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:49,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:49,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:49,399 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:49,399 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 00:54:49,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:49,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:49,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:49,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:49,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:49,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:49,401 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 00:54:49,401 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 00:54:49,401 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:49,401 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:49,401 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,401 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:49,403 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,403 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:49,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:49,403 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:49,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,403 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:49,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:49,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:49,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:49,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:49,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,404 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:49,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:49,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:49,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:49,406 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:49,406 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:49,406 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,406 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:49,406 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 00:54:49,406 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:49,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:49,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:49,406 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:49,406 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,407 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:49,407 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:49,407 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,407 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:49,407 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:49,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:49,407 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-13 00:54:49,407 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,507 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:49,508 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:49,512 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:49,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:49,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,514 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 00:54:49,514 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:49,514 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 00:54:49,514 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 00:54:49,514 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,514 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,515 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:49,515 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:49,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:49,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:49,516 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:49,516 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:49,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,516 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:49,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,516 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,516 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 00:54:49,516 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:49,517 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 00:54:49,517 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:49,517 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 00:54:49,517 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:49,517 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:49,517 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:49,517 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. started... 2023-02-13 00:54:49,517 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e740718b17018a47b245263e5bd100e3 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:49,929 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101d7110b87b6244e93bc2b9e95cf967ba6 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101d7110b87b6244e93bc2b9e95cf967ba6 2023-02-13 00:54:50,334 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/.tmp/family1/c4d477c410bb4c5186f30d0a03df6edc 2023-02-13 00:54:50,339 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/.tmp/family1/c4d477c410bb4c5186f30d0a03df6edc as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/c4d477c410bb4c5186f30d0a03df6edc 2023-02-13 00:54:50,343 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/c4d477c410bb4c5186f30d0a03df6edc, entries=1, sequenceid=29, filesize=5.0 K 2023-02-13 00:54:50,344 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e740718b17018a47b245263e5bd100e3 in 827ms, sequenceid=29, compaction requested=true 2023-02-13 00:54:50,344 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e740718b17018a47b245263e5bd100e3: 2023-02-13 00:54:50,344 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:50,344 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:50,344 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:50,344 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,344 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 00:54:50,344 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 00:54:50,347 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,347 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:50,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:50,347 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,347 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:50,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:50,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:50,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:50,349 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 00:54:50,349 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@72987305[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:50,349 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 00:54:50,349 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,350 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,350 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 00:54:50,350 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 00:54:50,350 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:50,350 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,350 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:50,351 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 00:54:50,351 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,351 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:50,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:50,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:50,351 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 00:54:50,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:50,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:50,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:50,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:50,353 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 00:54:50,353 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 00:54:50,353 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:50,353 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:50,353 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,353 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:50,358 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,358 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,358 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:50,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:50,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:50,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:50,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:50,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:50,358 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,377 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 00:54:50,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,377 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6d907849 to 127.0.0.1:54478 2023-02-13 00:54:50,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,378 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:54:50,378 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 00:54:50,378 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x53f4ef3a to 127.0.0.1:54478 2023-02-13 00:54:50,378 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:54:50,378 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 00:54:50,378 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7bfba985 to 127.0.0.1:54478 2023-02-13 00:54:50,379 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:54:50,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:50,380 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:50,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,382 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,382 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,382 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,382 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,382 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,382 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,382 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:50,382 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:50,382 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 00:54:50,382 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:50,382 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,382 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:50,383 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-13 00:54:50,382 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:50,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:50,383 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:50,383 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:50,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:50,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:50,483 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:50,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:50,488 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:50,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:50,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,490 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 00:54:50,490 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:50,490 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 00:54:50,490 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 00:54:50,490 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,490 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,493 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:50,493 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:50,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:50,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,494 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,494 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:50,494 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:50,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,495 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,495 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,495 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 00:54:50,495 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:50,495 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 00:54:50,497 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:50,497 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:50,497 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:50,497 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. started... 2023-02-13 00:54:50,497 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e740718b17018a47b245263e5bd100e3 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:50,917 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103d29bca72df6b4fcaaa5d1e9a83d5c02b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103d29bca72df6b4fcaaa5d1e9a83d5c02b 2023-02-13 00:54:50,923 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/.tmp/family1/c1b95fca90454f5cb0f223849dae9f8a 2023-02-13 00:54:50,928 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/.tmp/family1/c1b95fca90454f5cb0f223849dae9f8a as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/c1b95fca90454f5cb0f223849dae9f8a 2023-02-13 00:54:50,934 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/c1b95fca90454f5cb0f223849dae9f8a, entries=1, sequenceid=33, filesize=5.0 K 2023-02-13 00:54:50,935 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e740718b17018a47b245263e5bd100e3 in 438ms, sequenceid=33, compaction requested=true 2023-02-13 00:54:50,935 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e740718b17018a47b245263e5bd100e3: 2023-02-13 00:54:50,935 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:50,935 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:50,935 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:50,935 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,935 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 00:54:50,935 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 00:54:50,937 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,937 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:50,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:50,937 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,937 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:50,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:50,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:50,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:50,940 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 00:54:50,940 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2a89c36d[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:50,940 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 00:54:50,940 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,941 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,941 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 00:54:50,941 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 00:54:50,941 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:50,941 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,941 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:50,943 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 00:54:50,943 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,943 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:50,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:50,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:50,943 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 00:54:50,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:50,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:50,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:50,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:50,946 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 00:54:50,946 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:50,946 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:50,946 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 00:54:50,946 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,946 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:50,949 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,949 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,949 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:50,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:50,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:50,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,950 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:50,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:50,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:50,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:50,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,951 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:50,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,953 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,953 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:50,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:50,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:50,953 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,954 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,954 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:50,954 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,954 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,954 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:50,954 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:50,954 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 00:54:50,954 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:50,954 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:50,954 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-13 00:54:50,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:50,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:50,954 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:50,954 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:51,055 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:51,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:51,060 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:51,061 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:51,061 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,061 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 00:54:51,061 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:51,061 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 00:54:51,061 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 00:54:51,062 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,062 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,063 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:51,063 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:51,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:51,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:51,064 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:51,064 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:51,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,064 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:51,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,064 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,065 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 00:54:51,065 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,065 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 00:54:51,065 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:51,065 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 00:54:51,065 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:51,065 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:51,065 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:51,065 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. started... 2023-02-13 00:54:51,065 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e740718b17018a47b245263e5bd100e3 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:51,077 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090710a06b5a856646928aebc9f67d8b0235 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015090710a06b5a856646928aebc9f67d8b0235 2023-02-13 00:54:51,483 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/.tmp/family1/b52badb9cac64f17a139b3e0e79b068a 2023-02-13 00:54:51,488 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/.tmp/family1/b52badb9cac64f17a139b3e0e79b068a as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/b52badb9cac64f17a139b3e0e79b068a 2023-02-13 00:54:51,492 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/b52badb9cac64f17a139b3e0e79b068a, entries=1, sequenceid=37, filesize=5.0 K 2023-02-13 00:54:51,493 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e740718b17018a47b245263e5bd100e3 in 428ms, sequenceid=37, compaction requested=true 2023-02-13 00:54:51,493 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e740718b17018a47b245263e5bd100e3: 2023-02-13 00:54:51,493 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:51,493 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:51,493 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:51,493 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:51,493 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 00:54:51,493 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 00:54:51,495 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:51,495 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:51,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:51,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:51,495 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,495 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:51,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:51,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:51,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:51,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:51,496 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 00:54:51,496 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@406a1f93[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:51,496 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 00:54:51,497 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,499 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,499 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 00:54:51,499 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 00:54:51,499 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:51,499 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:51,499 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:51,500 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 00:54:51,500 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:51,500 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:51,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:51,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:51,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:51,500 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 00:54:51,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:51,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:51,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:51,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:51,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:51,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:51,503 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 00:54:51,503 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 00:54:51,503 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:51,503 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:51,503 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,503 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:51,504 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,504 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,504 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:51,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:51,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:51,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,505 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:51,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:51,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:51,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:51,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:51,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,506 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:51,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:51,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:51,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:51,518 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:51,518 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:51,518 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:51,518 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:51,518 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 00:54:51,518 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:51,518 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:51,518 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:51,519 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-13 00:54:51,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:51,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:51,518 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:51,519 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:51,519 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,519 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:51,519 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,519 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,619 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:51,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:51,623 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:51,625 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:51,625 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,625 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 00:54:51,625 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:51,625 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 00:54:51,625 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 00:54:51,626 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,626 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,627 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:51,627 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:51,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:51,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:51,627 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:51,627 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:51,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,628 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:51,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,628 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,628 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 00:54:51,628 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:51,628 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 00:54:51,628 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:51,628 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 00:54:51,629 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:51,629 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:51,629 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:51,629 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. started... 2023-02-13 00:54:51,629 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e740718b17018a47b245263e5bd100e3 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:54:52,041 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511202cf19ee0debc42e9899b9d00184bd655 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511202cf19ee0debc42e9899b9d00184bd655 2023-02-13 00:54:52,447 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/.tmp/family1/50d3504a2ae4408d88554bf62f8cb425 2023-02-13 00:54:52,452 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/.tmp/family1/50d3504a2ae4408d88554bf62f8cb425 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/50d3504a2ae4408d88554bf62f8cb425 2023-02-13 00:54:52,456 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/50d3504a2ae4408d88554bf62f8cb425, entries=1, sequenceid=41, filesize=5.0 K 2023-02-13 00:54:52,457 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e740718b17018a47b245263e5bd100e3 in 828ms, sequenceid=41, compaction requested=true 2023-02-13 00:54:52,457 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e740718b17018a47b245263e5bd100e3: 2023-02-13 00:54:52,457 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:52,457 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:52,457 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:52,457 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:52,457 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 00:54:52,457 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 00:54:52,459 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:52,460 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:52,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:52,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:52,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:52,460 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:52,460 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:52,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:52,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:52,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:52,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:52,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:52,461 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 00:54:52,461 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@22cf7911[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:52,461 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 00:54:52,461 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:52,462 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:52,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:52,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:52,462 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 00:54:52,462 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 00:54:52,463 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:54:52,463 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:52,463 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:52,464 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 00:54:52,464 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:52,464 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:52,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:52,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:52,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:52,464 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 00:54:52,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:52,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:52,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:52,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:52,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:52,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:52,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:52,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:54:52,466 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 00:54:52,466 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 00:54:52,466 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:52,466 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:52,466 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:52,466 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:52,468 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:52,468 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:52,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:52,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:52,468 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:52,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:52,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:52,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:52,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:52,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:52,468 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:52,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:52,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:52,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:52,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:52,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:52,469 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:52,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:52,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:52,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:52,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:52,473 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:52,473 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:52,473 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:52,473 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:52,473 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:52,473 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:52,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:52,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:52,473 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 00:54:52,473 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:52,473 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:52,473 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:52,473 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:52,473 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:52,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:52,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:52,473 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 00:54:52,473 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:52,573 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:52,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:52,575 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43563] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:52,577 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] procedure2.ProcedureExecutor(1029): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 00:54:52,578 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 00:54:52,578 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:54:52,578 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676250292578 2023-02-13 00:54:52,579 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 00:54:52,580 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-13 00:54:52,580 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 00:54:52,581 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] zookeeper.ReadOnlyZKClient(139): Connect 0x28ef2261 to 127.0.0.1:54478 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:54:52,585 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2d292ddf, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:54:52,585 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:53,422 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103db5ef6ad6d53454a86561199c5974940 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103db5ef6ad6d53454a86561199c5974940 2023-02-13 00:54:53,423 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 00:54:53,435 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:54:53,436 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50246, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:54:53,443 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 00:54:53,445 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57640, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 00:54:53,453 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/8bf441cfe7274b4597c62b0efdf616c2 first=Optional[r3] last=Optional[r6] 2023-02-13 00:54:53,454 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/8bf441cfe7274b4597c62b0efdf616c2}] 2023-02-13 00:54:53,458 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/8bf441cfe7274b4597c62b0efdf616c2 for inclusion in e740718b17018a47b245263e5bd100e3/family1 2023-02-13 00:54:53,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-13 00:54:53,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 00:54:53,461 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for e740718b17018a47b245263e5bd100e3: 2023-02-13 00:54:53,461 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/8bf441cfe7274b4597c62b0efdf616c2 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__25gl88v5qoqhgdkh8oq1qnahinhg3jrucn3kbp4ouloartrh3h0j7is6ujjcd5ah/family1/8bf441cfe7274b4597c62b0efdf616c2 2023-02-13 00:54:53,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__25gl88v5qoqhgdkh8oq1qnahinhg3jrucn3kbp4ouloartrh3h0j7is6ujjcd5ah/family1/8bf441cfe7274b4597c62b0efdf616c2 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/774c674638384eb0b614a66ce8a029cb_SeqId_44_ 2023-02-13 00:54:53,465 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__25gl88v5qoqhgdkh8oq1qnahinhg3jrucn3kbp4ouloartrh3h0j7is6ujjcd5ah/family1/8bf441cfe7274b4597c62b0efdf616c2 into e740718b17018a47b245263e5bd100e3/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/774c674638384eb0b614a66ce8a029cb_SeqId_44_ - updating store file list. 2023-02-13 00:54:53,470 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/774c674638384eb0b614a66ce8a029cb_SeqId_44_ into e740718b17018a47b245263e5bd100e3/family1 2023-02-13 00:54:53,470 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__25gl88v5qoqhgdkh8oq1qnahinhg3jrucn3kbp4ouloartrh3h0j7is6ujjcd5ah/family1/8bf441cfe7274b4597c62b0efdf616c2 into e740718b17018a47b245263e5bd100e3/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/774c674638384eb0b614a66ce8a029cb_SeqId_44_) 2023-02-13 00:54:53,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__25gl88v5qoqhgdkh8oq1qnahinhg3jrucn3kbp4ouloartrh3h0j7is6ujjcd5ah/family1/8bf441cfe7274b4597c62b0efdf616c2 2023-02-13 00:54:53,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-13 00:54:53,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-13 00:54:53,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(1912): e740718b17018a47b245263e5bd100e3/family1 is initiating minor compaction 2023-02-13 00:54:53,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@13cfd8af; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:53,484 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of e740718b17018a47b245263e5bd100e3/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:53,484 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/4a7dd7f4523148e0b934b9f3e6e8dd91, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/e6e9c8640c1a404cb5dd8912b91d3af5, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/af03cd173d8f495890da469d50448a4e, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/6f035f69746f41d0964b8ca16f86cc35, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/4b1e45d7d59a47d8b587a179910afe45, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/c4d477c410bb4c5186f30d0a03df6edc, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/c1b95fca90454f5cb0f223849dae9f8a, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/b52badb9cac64f17a139b3e0e79b068a, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/50d3504a2ae4408d88554bf62f8cb425, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/774c674638384eb0b614a66ce8a029cb_SeqId_44_] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/.tmp, totalSize=50.4 K 2023-02-13 00:54:53,484 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 4a7dd7f4523148e0b934b9f3e6e8dd91, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-13 00:54:53,485 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting e6e9c8640c1a404cb5dd8912b91d3af5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-13 00:54:53,485 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting af03cd173d8f495890da469d50448a4e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-13 00:54:53,485 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 6f035f69746f41d0964b8ca16f86cc35, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-13 00:54:53,485 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 4b1e45d7d59a47d8b587a179910afe45, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-13 00:54:53,485 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting c4d477c410bb4c5186f30d0a03df6edc, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-13 00:54:53,485 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting c1b95fca90454f5cb0f223849dae9f8a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-13 00:54:53,485 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting b52badb9cac64f17a139b3e0e79b068a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-13 00:54:53,485 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 50d3504a2ae4408d88554bf62f8cb425, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-13 00:54:53,485 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 774c674638384eb0b614a66ce8a029cb_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-13 00:54:53,488 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 00:54:53,489 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:53,491 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512283a79ae107c4b46169f5d32553364f1f7 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512283a79ae107c4b46169f5d32553364f1f7 2023-02-13 00:54:53,493 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015123169d57d5cb14b476882ee86e9413098a3 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015123169d57d5cb14b476882ee86e9413098a3 2023-02-13 00:54:53,494 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101d7110b87b6244e93bc2b9e95cf967ba6 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101d7110b87b6244e93bc2b9e95cf967ba6 2023-02-13 00:54:53,496 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103d29bca72df6b4fcaaa5d1e9a83d5c02b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103d29bca72df6b4fcaaa5d1e9a83d5c02b 2023-02-13 00:54:53,496 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-02-13 00:54:53,496 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-13 00:54:53,497 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:53,497 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:53,497 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:53,497 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:53,497 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:53,497 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:53,497 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:53,497 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:53,523 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] throttle.PressureAwareThroughputController(145): e740718b17018a47b245263e5bd100e3#family1#compaction#60 average throughput is 0.79 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:54:54,349 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205c1d395b3b5c2475aa92720bc116673c2 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205c1d395b3b5c2475aa92720bc116673c2 2023-02-13 00:54:54,351 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 00:54:54,351 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/.tmp/family1/0dcdd7b2d48b41aeb76695825355a069 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/0dcdd7b2d48b41aeb76695825355a069 2023-02-13 00:54:54,355 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in e740718b17018a47b245263e5bd100e3/family1 of e740718b17018a47b245263e5bd100e3 into 0dcdd7b2d48b41aeb76695825355a069(size=6.8 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:54:54,355 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for e740718b17018a47b245263e5bd100e3: 2023-02-13 00:54:54,355 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1676249693484; duration=0sec 2023-02-13 00:54:54,355 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:54,379 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b89d44b7ff734edd8bfe5189197359e5 first=Optional[r00] last=Optional[r2] 2023-02-13 00:54:54,380 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b89d44b7ff734edd8bfe5189197359e5}] 2023-02-13 00:54:54,382 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b89d44b7ff734edd8bfe5189197359e5 for inclusion in e740718b17018a47b245263e5bd100e3/family1 2023-02-13 00:54:54,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-13 00:54:54,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 00:54:54,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for e740718b17018a47b245263e5bd100e3: 2023-02-13 00:54:54,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b89d44b7ff734edd8bfe5189197359e5 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__6ril6hgrejkokhqkc8vintf26r8k8f9f5ucp3lb2eu7kb1o9i883jeqtnhsh4bn/family1/b89d44b7ff734edd8bfe5189197359e5 2023-02-13 00:54:54,387 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__6ril6hgrejkokhqkc8vintf26r8k8f9f5ucp3lb2eu7kb1o9i883jeqtnhsh4bn/family1/b89d44b7ff734edd8bfe5189197359e5 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/f5eff8d8182b426487c7b3ef22b2d9ec_SeqId_47_ 2023-02-13 00:54:54,387 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__6ril6hgrejkokhqkc8vintf26r8k8f9f5ucp3lb2eu7kb1o9i883jeqtnhsh4bn/family1/b89d44b7ff734edd8bfe5189197359e5 into e740718b17018a47b245263e5bd100e3/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/f5eff8d8182b426487c7b3ef22b2d9ec_SeqId_47_ - updating store file list. 2023-02-13 00:54:54,390 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/f5eff8d8182b426487c7b3ef22b2d9ec_SeqId_47_ into e740718b17018a47b245263e5bd100e3/family1 2023-02-13 00:54:54,391 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__6ril6hgrejkokhqkc8vintf26r8k8f9f5ucp3lb2eu7kb1o9i883jeqtnhsh4bn/family1/b89d44b7ff734edd8bfe5189197359e5 into e740718b17018a47b245263e5bd100e3/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/f5eff8d8182b426487c7b3ef22b2d9ec_SeqId_47_) 2023-02-13 00:54:54,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__6ril6hgrejkokhqkc8vintf26r8k8f9f5ucp3lb2eu7kb1o9i883jeqtnhsh4bn/family1/b89d44b7ff734edd8bfe5189197359e5 2023-02-13 00:54:54,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-13 00:54:54,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-13 00:54:54,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(1912): e740718b17018a47b245263e5bd100e3/family1 is initiating minor compaction (all files) 2023-02-13 00:54:54,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3d9535db; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:54,392 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of e740718b17018a47b245263e5bd100e3/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:54,392 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/17a477ffebe040ccb4e5183f637fe0d1, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/0dcdd7b2d48b41aeb76695825355a069, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/f5eff8d8182b426487c7b3ef22b2d9ec_SeqId_47_] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/.tmp, totalSize=390.1 K 2023-02-13 00:54:54,393 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 17a477ffebe040ccb4e5183f637fe0d1, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878845884 2023-02-13 00:54:54,393 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 0dcdd7b2d48b41aeb76695825355a069, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621245884 2023-02-13 00:54:54,393 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting f5eff8d8182b426487c7b3ef22b2d9ec_SeqId_47_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878845884 2023-02-13 00:54:54,394 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 00:54:54,394 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:54,396 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130a532a233d4b3487b8e2d4199e1e87b67 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130a532a233d4b3487b8e2d4199e1e87b67 2023-02-13 00:54:54,398 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201c1ed825581454c0eaab0712eb6e80cd2 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201c1ed825581454c0eaab0712eb6e80cd2 2023-02-13 00:54:54,399 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120560f163a7892348b4a2c9a955fe6d3c3a to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120560f163a7892348b4a2c9a955fe6d3c3a 2023-02-13 00:54:54,399 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-02-13 00:54:54,399 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-13 00:54:54,399 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:54,400 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 00:54:54,400 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:54:54,401 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 00:54:54,402 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.8250 sec 2023-02-13 00:54:54,408 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] throttle.PressureAwareThroughputController(145): e740718b17018a47b245263e5bd100e3#family1#compaction#61 average throughput is 30.10 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:54:54,455 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/4a7dd7f4523148e0b934b9f3e6e8dd91, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/e6e9c8640c1a404cb5dd8912b91d3af5, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/af03cd173d8f495890da469d50448a4e, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/6f035f69746f41d0964b8ca16f86cc35, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/4b1e45d7d59a47d8b587a179910afe45, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/c4d477c410bb4c5186f30d0a03df6edc, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/c1b95fca90454f5cb0f223849dae9f8a, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/b52badb9cac64f17a139b3e0e79b068a, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/50d3504a2ae4408d88554bf62f8cb425, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/774c674638384eb0b614a66ce8a029cb_SeqId_44_] to archive 2023-02-13 00:54:54,456 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:54,458 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/4a7dd7f4523148e0b934b9f3e6e8dd91 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/4a7dd7f4523148e0b934b9f3e6e8dd91 2023-02-13 00:54:54,461 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/e6e9c8640c1a404cb5dd8912b91d3af5 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/e6e9c8640c1a404cb5dd8912b91d3af5 2023-02-13 00:54:54,463 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/af03cd173d8f495890da469d50448a4e to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/af03cd173d8f495890da469d50448a4e 2023-02-13 00:54:54,465 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/6f035f69746f41d0964b8ca16f86cc35 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/6f035f69746f41d0964b8ca16f86cc35 2023-02-13 00:54:54,466 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/4b1e45d7d59a47d8b587a179910afe45 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/4b1e45d7d59a47d8b587a179910afe45 2023-02-13 00:54:54,467 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/c4d477c410bb4c5186f30d0a03df6edc to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/c4d477c410bb4c5186f30d0a03df6edc 2023-02-13 00:54:54,469 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/c1b95fca90454f5cb0f223849dae9f8a to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/c1b95fca90454f5cb0f223849dae9f8a 2023-02-13 00:54:54,471 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/b52badb9cac64f17a139b3e0e79b068a to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/b52badb9cac64f17a139b3e0e79b068a 2023-02-13 00:54:54,472 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/50d3504a2ae4408d88554bf62f8cb425 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/50d3504a2ae4408d88554bf62f8cb425 2023-02-13 00:54:54,474 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/774c674638384eb0b614a66ce8a029cb_SeqId_44_ to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/774c674638384eb0b614a66ce8a029cb_SeqId_44_ 2023-02-13 00:54:54,492 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-13 00:54:54,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure2.ProcedureExecutor(1029): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:54,902 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-13 00:54:54,903 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-02-13 00:54:54,903 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-13 00:54:54,905 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-02-13 00:54:54,908 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=e740718b17018a47b245263e5bd100e3, REOPEN/MOVE}] 2023-02-13 00:54:54,908 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=e740718b17018a47b245263e5bd100e3, REOPEN/MOVE 2023-02-13 00:54:54,909 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=e740718b17018a47b245263e5bd100e3, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:54,909 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676249694909"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249694909"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249694909"}]},"ts":"1676249694909"} 2023-02-13 00:54:54,910 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure e740718b17018a47b245263e5bd100e3, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:54:55,062 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:55,062 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e740718b17018a47b245263e5bd100e3, disabling compactions & flushes 2023-02-13 00:54:55,062 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:55,625 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/.tmp/family1/9355456b054844f1ba591bee13aa3cd2 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/9355456b054844f1ba591bee13aa3cd2 2023-02-13 00:54:55,629 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in e740718b17018a47b245263e5bd100e3/family1 of e740718b17018a47b245263e5bd100e3 into 9355456b054844f1ba591bee13aa3cd2(size=193.0 K), total size for store is 193.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 00:54:55,629 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for e740718b17018a47b245263e5bd100e3: 2023-02-13 00:54:55,629 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:55,629 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3., storeName=family1, fileCount=3, fileSize=390.1 K (191.4 K, 6.8 K, 191.8 K), priority=2, time=1676249694392; duration=1sec 2023-02-13 00:54:55,629 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:55,629 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. after waiting 0 ms 2023-02-13 00:54:55,629 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:55,629 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:55,630 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/17a477ffebe040ccb4e5183f637fe0d1, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/0dcdd7b2d48b41aeb76695825355a069, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/f5eff8d8182b426487c7b3ef22b2d9ec_SeqId_47_] to archive 2023-02-13 00:54:55,630 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:55,632 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/17a477ffebe040ccb4e5183f637fe0d1 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/17a477ffebe040ccb4e5183f637fe0d1 2023-02-13 00:54:55,633 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/0dcdd7b2d48b41aeb76695825355a069 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/0dcdd7b2d48b41aeb76695825355a069 2023-02-13 00:54:55,634 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/f5eff8d8182b426487c7b3ef22b2d9ec_SeqId_47_ to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/f5eff8d8182b426487c7b3ef22b2d9ec_SeqId_47_ 2023-02-13 00:54:55,639 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-13 00:54:55,639 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:55,639 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e740718b17018a47b245263e5bd100e3: Waiting for close lock at 1676249695062Running coprocessor pre-close hooks at 1676249695062Disabling compacts and flushes for region at 1676249695062Disabling writes for close at 1676249695629 (+567 ms)Writing region close event to WAL at 1676249695636 (+7 ms)Running coprocessor post-close hooks at 1676249695639 (+3 ms)Closed at 1676249695639 2023-02-13 00:54:55,639 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: e740718b17018a47b245263e5bd100e3 to self. 2023-02-13 00:54:55,641 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:55,641 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=e740718b17018a47b245263e5bd100e3, regionState=CLOSED 2023-02-13 00:54:55,641 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676249695641"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249695641"}]},"ts":"1676249695641"} 2023-02-13 00:54:55,643 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-13 00:54:55,643 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure e740718b17018a47b245263e5bd100e3, server=jenkins-hbase4.apache.org,36605,1676249622318 in 732 msec 2023-02-13 00:54:55,644 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=e740718b17018a47b245263e5bd100e3, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=true 2023-02-13 00:54:55,794 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=e740718b17018a47b245263e5bd100e3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:55,794 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676249695794"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249695794"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249695794"}]},"ts":"1676249695794"} 2023-02-13 00:54:55,796 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure e740718b17018a47b245263e5bd100e3, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:54:55,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:55,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e740718b17018a47b245263e5bd100e3, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:54:55,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:55,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:55,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:55,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:55,953 INFO [StoreOpener-e740718b17018a47b245263e5bd100e3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:55,954 DEBUG [StoreOpener-e740718b17018a47b245263e5bd100e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1 2023-02-13 00:54:55,954 DEBUG [StoreOpener-e740718b17018a47b245263e5bd100e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1 2023-02-13 00:54:55,954 INFO [StoreOpener-e740718b17018a47b245263e5bd100e3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e740718b17018a47b245263e5bd100e3 columnFamilyName family1 2023-02-13 00:54:55,959 DEBUG [StoreOpener-e740718b17018a47b245263e5bd100e3-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/9355456b054844f1ba591bee13aa3cd2 2023-02-13 00:54:55,960 INFO [StoreOpener-e740718b17018a47b245263e5bd100e3-1] regionserver.HStore(310): Store=e740718b17018a47b245263e5bd100e3/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:55,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:55,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:55,964 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:55,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e740718b17018a47b245263e5bd100e3; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70953853, jitterRate=0.057294800877571106}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 00:54:55,965 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e740718b17018a47b245263e5bd100e3: Running coprocessor pre-open hook at 1676249695951Writing region info on filesystem at 1676249695951Initializing all the Stores at 1676249695952 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1676249695952Cleaning up temporary data from old regions at 1676249695962 (+10 ms)Cleaning up detritus from prior splits at 1676249695962Running coprocessor post-open hooks at 1676249695965 (+3 ms)Region opened successfully at 1676249695965 2023-02-13 00:54:55,966 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3., pid=47, masterSystemTime=1676249695947 2023-02-13 00:54:55,967 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:55,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:55,968 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=e740718b17018a47b245263e5bd100e3, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:55,968 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676249695968"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249695968"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249695968"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249695968"}]},"ts":"1676249695968"} 2023-02-13 00:54:55,970 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=47, resume processing ppid=45 2023-02-13 00:54:55,971 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure e740718b17018a47b245263e5bd100e3, server=jenkins-hbase4.apache.org,36605,1676249622318 in 173 msec 2023-02-13 00:54:55,972 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-13 00:54:55,972 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=e740718b17018a47b245263e5bd100e3, REOPEN/MOVE in 1.0620 sec 2023-02-13 00:54:55,974 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=43 2023-02-13 00:54:55,974 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.0680 sec 2023-02-13 00:54:55,975 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.4830 sec 2023-02-13 00:54:55,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=43 2023-02-13 00:54:55,977 INFO [Listener at localhost/37869] compactions.TestMobCompactor(278): alter status finished 2023-02-13 00:54:55,978 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43563] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:55,979 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] procedure2.ProcedureExecutor(1029): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 00:54:55,980 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 00:54:55,980 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676250295980 2023-02-13 00:54:55,980 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:54:55,981 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 00:54:55,981 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-13 00:54:55,981 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 00:54:55,982 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] zookeeper.ReadOnlyZKClient(139): Connect 0x11ed5c11 to 127.0.0.1:54478 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:54:55,989 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@754edc26, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:54:55,990 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:55,990 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:55,990 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:55,990 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:55,990 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:56,821 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112859e269c98c984dc1b6560e14b131b31e to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112859e269c98c984dc1b6560e14b131b31e 2023-02-13 00:54:56,822 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 00:54:56,837 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:54:56,838 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50256, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:54:56,846 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 00:54:56,848 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57652, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 00:54:56,858 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/30fab39ac837406db14f92d2c1554647 first=Optional[r05] last=Optional[r8] 2023-02-13 00:54:56,872 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=51 for row with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/30fab39ac837406db14f92d2c1554647}] 2023-02-13 00:54:56,878 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/30fab39ac837406db14f92d2c1554647 for inclusion in e740718b17018a47b245263e5bd100e3/family1 2023-02-13 00:54:56,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-13 00:54:56,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 00:54:56,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for e740718b17018a47b245263e5bd100e3: 2023-02-13 00:54:56,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/30fab39ac837406db14f92d2c1554647 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__g4oa728g4jp6qi7rt9oi6faam4ri9vm7c35euubrreq0nra6i4tpbvl23nivrpbk/family1/30fab39ac837406db14f92d2c1554647 2023-02-13 00:54:56,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__g4oa728g4jp6qi7rt9oi6faam4ri9vm7c35euubrreq0nra6i4tpbvl23nivrpbk/family1/30fab39ac837406db14f92d2c1554647 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/4e708be4d022439583b0e66958533fdc_SeqId_53_ 2023-02-13 00:54:56,885 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__g4oa728g4jp6qi7rt9oi6faam4ri9vm7c35euubrreq0nra6i4tpbvl23nivrpbk/family1/30fab39ac837406db14f92d2c1554647 into e740718b17018a47b245263e5bd100e3/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/4e708be4d022439583b0e66958533fdc_SeqId_53_ - updating store file list. 2023-02-13 00:54:56,889 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/4e708be4d022439583b0e66958533fdc_SeqId_53_ into e740718b17018a47b245263e5bd100e3/family1 2023-02-13 00:54:56,889 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__g4oa728g4jp6qi7rt9oi6faam4ri9vm7c35euubrreq0nra6i4tpbvl23nivrpbk/family1/30fab39ac837406db14f92d2c1554647 into e740718b17018a47b245263e5bd100e3/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/4e708be4d022439583b0e66958533fdc_SeqId_53_) 2023-02-13 00:54:56,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__g4oa728g4jp6qi7rt9oi6faam4ri9vm7c35euubrreq0nra6i4tpbvl23nivrpbk/family1/30fab39ac837406db14f92d2c1554647 2023-02-13 00:54:56,892 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 00:54:56,892 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:56,894 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511202cf19ee0debc42e9899b9d00184bd655 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511202cf19ee0debc42e9899b9d00184bd655 2023-02-13 00:54:56,895 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128027c570cbc9248b48445dabfb4c5df52 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128027c570cbc9248b48445dabfb4c5df52 2023-02-13 00:54:56,895 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-02-13 00:54:56,895 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-13 00:54:56,896 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:56,896 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:56,896 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:56,896 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 00:54:56,896 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:54:56,897 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 00:54:56,898 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 918 msec 2023-02-13 00:54:56,953 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-13 00:54:56,954 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:57,364 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-02-13 00:54:57,365 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-13 00:54:57,365 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-02-13 00:54:57,367 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-02-13 00:54:57,370 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=e740718b17018a47b245263e5bd100e3, REOPEN/MOVE}] 2023-02-13 00:54:57,370 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=e740718b17018a47b245263e5bd100e3, REOPEN/MOVE 2023-02-13 00:54:57,371 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=e740718b17018a47b245263e5bd100e3, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:57,371 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676249697371"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249697371"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249697371"}]},"ts":"1676249697371"} 2023-02-13 00:54:57,373 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure e740718b17018a47b245263e5bd100e3, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:54:57,524 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:57,524 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e740718b17018a47b245263e5bd100e3, disabling compactions & flushes 2023-02-13 00:54:57,525 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:57,525 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:57,525 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. after waiting 0 ms 2023-02-13 00:54:57,525 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:57,530 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-13 00:54:57,531 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:57,531 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e740718b17018a47b245263e5bd100e3: Waiting for close lock at 1676249697524Running coprocessor pre-close hooks at 1676249697524Disabling compacts and flushes for region at 1676249697524Disabling writes for close at 1676249697525 (+1 ms)Writing region close event to WAL at 1676249697527 (+2 ms)Running coprocessor post-close hooks at 1676249697530 (+3 ms)Closed at 1676249697530 2023-02-13 00:54:57,531 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: e740718b17018a47b245263e5bd100e3 to self. 2023-02-13 00:54:57,532 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:57,532 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=e740718b17018a47b245263e5bd100e3, regionState=CLOSED 2023-02-13 00:54:57,532 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676249697532"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249697532"}]},"ts":"1676249697532"} 2023-02-13 00:54:57,535 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=52, resume processing ppid=51 2023-02-13 00:54:57,535 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure e740718b17018a47b245263e5bd100e3, server=jenkins-hbase4.apache.org,36605,1676249622318 in 160 msec 2023-02-13 00:54:57,535 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=e740718b17018a47b245263e5bd100e3, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=true 2023-02-13 00:54:57,685 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=e740718b17018a47b245263e5bd100e3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:57,686 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676249697685"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249697685"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249697685"}]},"ts":"1676249697685"} 2023-02-13 00:54:57,687 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure e740718b17018a47b245263e5bd100e3, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:54:57,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:57,841 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e740718b17018a47b245263e5bd100e3, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:54:57,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:57,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:57,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:57,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:57,843 INFO [StoreOpener-e740718b17018a47b245263e5bd100e3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:57,844 DEBUG [StoreOpener-e740718b17018a47b245263e5bd100e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1 2023-02-13 00:54:57,844 DEBUG [StoreOpener-e740718b17018a47b245263e5bd100e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1 2023-02-13 00:54:57,844 INFO [StoreOpener-e740718b17018a47b245263e5bd100e3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e740718b17018a47b245263e5bd100e3 columnFamilyName family1 2023-02-13 00:54:57,849 DEBUG [StoreOpener-e740718b17018a47b245263e5bd100e3-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/4e708be4d022439583b0e66958533fdc_SeqId_53_ 2023-02-13 00:54:57,851 DEBUG [StoreOpener-e740718b17018a47b245263e5bd100e3-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/family1/9355456b054844f1ba591bee13aa3cd2 2023-02-13 00:54:57,852 INFO [StoreOpener-e740718b17018a47b245263e5bd100e3-1] regionserver.HStore(310): Store=e740718b17018a47b245263e5bd100e3/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:57,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:57,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:57,856 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e740718b17018a47b245263e5bd100e3 2023-02-13 00:54:57,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e740718b17018a47b245263e5bd100e3; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61021077, jitterRate=-0.09071509540081024}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 00:54:57,857 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e740718b17018a47b245263e5bd100e3: Running coprocessor pre-open hook at 1676249697842Writing region info on filesystem at 1676249697842Initializing all the Stores at 1676249697843 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676249697843Cleaning up temporary data from old regions at 1676249697854 (+11 ms)Cleaning up detritus from prior splits at 1676249697854Running coprocessor post-open hooks at 1676249697856 (+2 ms)Region opened successfully at 1676249697856 2023-02-13 00:54:57,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3., pid=53, masterSystemTime=1676249697839 2023-02-13 00:54:57,858 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:57,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:54:57,859 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=e740718b17018a47b245263e5bd100e3, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:57,859 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676249697859"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249697859"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249697859"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249697859"}]},"ts":"1676249697859"} 2023-02-13 00:54:57,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=53, resume processing ppid=51 2023-02-13 00:54:57,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure e740718b17018a47b245263e5bd100e3, server=jenkins-hbase4.apache.org,36605,1676249622318 in 173 msec 2023-02-13 00:54:57,863 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-13 00:54:57,863 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=e740718b17018a47b245263e5bd100e3, REOPEN/MOVE in 492 msec 2023-02-13 00:54:57,865 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-13 00:54:57,865 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 497 msec 2023-02-13 00:54:57,867 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 913 msec 2023-02-13 00:54:57,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-13 00:54:57,869 INFO [Listener at localhost/37869] compactions.TestMobCompactor(278): alter status finished 2023-02-13 00:54:57,870 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43563] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 00:54:57,871 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] procedure2.ProcedureExecutor(1029): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 00:54:57,872 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 00:54:57,873 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:54:57,873 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676250297873 2023-02-13 00:54:57,874 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 00:54:57,874 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-13 00:54:57,874 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 00:54:57,874 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] zookeeper.ReadOnlyZKClient(139): Connect 0x188a36eb to 127.0.0.1:54478 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:54:57,879 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1b266630, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:54:57,879 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:57,879 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:57,880 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:57,880 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:57,880 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:57,880 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:57,880 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:57,880 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 00:54:57,880 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 00:54:57,880 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:54:57,881 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 00:54:57,882 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 10 msec 2023-02-13 00:54:57,938 INFO [Listener at localhost/37869] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=245 (was 260), OpenFileDescriptor=620 (was 629), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=233 (was 216) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 168), AvailableMemoryMB=3501 (was 3818) 2023-02-13 00:54:57,948 INFO [Listener at localhost/37869] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=245, OpenFileDescriptor=620, MaxFileDescriptor=60000, SystemLoadAverage=233, ProcessCount=167, AvailableMemoryMB=3501 2023-02-13 00:54:57,950 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-13 00:54:57,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure2.ProcedureExecutor(1029): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:54:57,952 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 00:54:57,952 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-02-13 00:54:57,952 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 00:54:57,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-13 00:54:57,953 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:54:57,954 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a empty. 2023-02-13 00:54:57,954 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:54:57,954 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-02-13 00:54:58,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-13 00:54:58,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-13 00:54:58,361 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-02-13 00:54:58,363 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7671): creating {ENCODED => cfb81dcb75adebd110154b74cfdaab0a, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp 2023-02-13 00:54:58,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-13 00:54:58,768 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:58,769 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1603): Closing cfb81dcb75adebd110154b74cfdaab0a, disabling compactions & flushes 2023-02-13 00:54:58,769 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:54:58,769 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:54:58,769 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. after waiting 0 ms 2023-02-13 00:54:58,769 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:54:58,769 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:54:58,769 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1557): Region close journal for cfb81dcb75adebd110154b74cfdaab0a: Waiting for close lock at 1676249698769Disabling compacts and flushes for region at 1676249698769Disabling writes for close at 1676249698769Writing region close event to WAL at 1676249698769Closed at 1676249698769 2023-02-13 00:54:58,771 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 00:54:58,772 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676249698771"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249698771"}]},"ts":"1676249698771"} 2023-02-13 00:54:58,773 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 00:54:58,773 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 00:54:58,773 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249698773"}]},"ts":"1676249698773"} 2023-02-13 00:54:58,774 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-02-13 00:54:58,777 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=cfb81dcb75adebd110154b74cfdaab0a, ASSIGN}] 2023-02-13 00:54:58,778 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=cfb81dcb75adebd110154b74cfdaab0a, ASSIGN 2023-02-13 00:54:58,778 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=cfb81dcb75adebd110154b74cfdaab0a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=false 2023-02-13 00:54:58,929 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=cfb81dcb75adebd110154b74cfdaab0a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:58,930 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676249698929"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249698929"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249698929"}]},"ts":"1676249698929"} 2023-02-13 00:54:58,931 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure cfb81dcb75adebd110154b74cfdaab0a, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:54:59,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-13 00:54:59,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:54:59,086 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => cfb81dcb75adebd110154b74cfdaab0a, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:54:59,086 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:54:59,086 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:59,086 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:54:59,086 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:54:59,087 INFO [StoreOpener-cfb81dcb75adebd110154b74cfdaab0a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:54:59,088 DEBUG [StoreOpener-cfb81dcb75adebd110154b74cfdaab0a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1 2023-02-13 00:54:59,088 DEBUG [StoreOpener-cfb81dcb75adebd110154b74cfdaab0a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1 2023-02-13 00:54:59,089 INFO [StoreOpener-cfb81dcb75adebd110154b74cfdaab0a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cfb81dcb75adebd110154b74cfdaab0a columnFamilyName family1 2023-02-13 00:54:59,090 INFO [StoreOpener-cfb81dcb75adebd110154b74cfdaab0a-1] regionserver.HStore(310): Store=cfb81dcb75adebd110154b74cfdaab0a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:59,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:54:59,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:54:59,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:54:59,095 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:54:59,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened cfb81dcb75adebd110154b74cfdaab0a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66144032, jitterRate=-0.014377117156982422}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 00:54:59,095 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for cfb81dcb75adebd110154b74cfdaab0a: Running coprocessor pre-open hook at 1676249699086Writing region info on filesystem at 1676249699086Initializing all the Stores at 1676249699087 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676249699087Cleaning up temporary data from old regions at 1676249699091 (+4 ms)Cleaning up detritus from prior splits at 1676249699092 (+1 ms)Running coprocessor post-open hooks at 1676249699095 (+3 ms)Region opened successfully at 1676249699095 2023-02-13 00:54:59,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a., pid=57, masterSystemTime=1676249699083 2023-02-13 00:54:59,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:54:59,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:54:59,098 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=cfb81dcb75adebd110154b74cfdaab0a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:54:59,098 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676249699098"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249699098"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249699098"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249699098"}]},"ts":"1676249699098"} 2023-02-13 00:54:59,100 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-13 00:54:59,100 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure cfb81dcb75adebd110154b74cfdaab0a, server=jenkins-hbase4.apache.org,36605,1676249622318 in 168 msec 2023-02-13 00:54:59,101 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=55 2023-02-13 00:54:59,101 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=cfb81dcb75adebd110154b74cfdaab0a, ASSIGN in 323 msec 2023-02-13 00:54:59,102 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 00:54:59,102 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249699102"}]},"ts":"1676249699102"} 2023-02-13 00:54:59,103 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-02-13 00:54:59,106 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 00:54:59,107 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1560 sec 2023-02-13 00:55:00,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-13 00:55:00,057 INFO [Listener at localhost/37869] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-02-13 00:55:00,083 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:00,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,085 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 00:55:00,085 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:00,085 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 00:55:00,085 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 00:55:00,085 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,085 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,087 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:00,087 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:00,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:00,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:00,088 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:00,088 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:00,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,088 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:00,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,089 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,089 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,089 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 00:55:00,089 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:00,089 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 00:55:00,089 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:55:00,089 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:00,089 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:00,089 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. started... 2023-02-13 00:55:00,090 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing cfb81dcb75adebd110154b74cfdaab0a 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-13 00:55:00,511 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130cdc47d88ec8c44eb9471152e592e8566 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130cdc47d88ec8c44eb9471152e592e8566 2023-02-13 00:55:00,515 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/.tmp/family1/d481003d34da46c388519b8457624e6c 2023-02-13 00:55:00,520 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/.tmp/family1/d481003d34da46c388519b8457624e6c as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/d481003d34da46c388519b8457624e6c 2023-02-13 00:55:00,524 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/d481003d34da46c388519b8457624e6c, entries=1000, sequenceid=5, filesize=190.5 K 2023-02-13 00:55:00,525 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for cfb81dcb75adebd110154b74cfdaab0a in 435ms, sequenceid=5, compaction requested=false 2023-02-13 00:55:00,525 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 00:55:00,526 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for cfb81dcb75adebd110154b74cfdaab0a: 2023-02-13 00:55:00,526 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:00,526 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:55:00,526 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:55:00,526 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:00,526 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 00:55:00,526 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 00:55:00,527 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,527 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:00,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:00,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:00,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:00,528 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,528 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:00,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:00,529 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 00:55:00,529 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@13fe2616[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:00,529 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 00:55:00,529 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,530 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,530 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 00:55:00,530 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 00:55:00,530 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:55:00,530 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:00,530 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:00,533 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 00:55:00,533 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:00,533 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:00,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:00,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:00,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:00,533 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 00:55:00,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:00,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:00,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:00,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:00,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:00,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:55:00,535 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 00:55:00,535 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 00:55:00,535 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:00,535 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:00,535 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,535 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:00,536 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,536 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:00,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:00,536 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:00,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,537 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:00,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:00,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:00,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:00,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:00,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,538 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:00,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:00,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:00,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:00,543 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:00,543 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:00,543 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:00,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:00,543 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:00,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:00,543 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 00:55:00,543 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:00,543 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,543 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,543 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:00,543 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,543 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:00,543 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:00,543 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 00:55:00,543 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:00,643 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:00,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:00,649 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:00,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 00:55:00,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,651 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 00:55:00,651 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:00,651 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 00:55:00,651 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 00:55:00,651 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,651 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,653 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:00,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:00,654 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:00,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:00,654 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:00,654 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:00,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,655 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:00,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,655 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,655 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 00:55:00,655 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:00,655 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 00:55:00,655 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:00,656 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 00:55:00,656 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:55:00,656 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:00,656 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:00,656 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. started... 2023-02-13 00:55:00,656 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing cfb81dcb75adebd110154b74cfdaab0a 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-13 00:55:01,078 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128e05ec97c1a05428f8463cf3c4b0e7930 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128e05ec97c1a05428f8463cf3c4b0e7930 2023-02-13 00:55:01,177 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 00:55:01,484 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/.tmp/family1/87ab412cb9694811843cfeaa4b1dd9a9 2023-02-13 00:55:01,489 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/.tmp/family1/87ab412cb9694811843cfeaa4b1dd9a9 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/87ab412cb9694811843cfeaa4b1dd9a9 2023-02-13 00:55:01,493 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/87ab412cb9694811843cfeaa4b1dd9a9, entries=1, sequenceid=9, filesize=5.0 K 2023-02-13 00:55:01,494 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for cfb81dcb75adebd110154b74cfdaab0a in 838ms, sequenceid=9, compaction requested=false 2023-02-13 00:55:01,494 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for cfb81dcb75adebd110154b74cfdaab0a: 2023-02-13 00:55:01,494 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:01,494 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:55:01,494 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:55:01,494 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:01,494 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 00:55:01,494 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 00:55:01,496 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,496 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:01,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:01,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:01,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:01,496 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,496 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:01,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:01,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:01,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:01,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:01,497 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 00:55:01,498 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@77081daf[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:01,498 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 00:55:01,498 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,499 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,499 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 00:55:01,499 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 00:55:01,499 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:55:01,499 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:01,499 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:01,500 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:01,500 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 00:55:01,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:01,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:01,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:01,501 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:01,501 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 00:55:01,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:01,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:01,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:01,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:01,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:01,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:55:01,503 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 00:55:01,503 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 00:55:01,503 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:01,503 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:01,503 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,503 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:01,506 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,506 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,506 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:01,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:01,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:01,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:01,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:01,507 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:01,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:01,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:01,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,508 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:01,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:01,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:01,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:01,510 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:01,510 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:01,510 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:01,510 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:01,510 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,510 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:01,510 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:01,510 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 00:55:01,511 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:01,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:01,510 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,511 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,511 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,511 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 00:55:01,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:01,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:01,511 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:01,611 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:01,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:01,616 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:01,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 00:55:01,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,618 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 00:55:01,618 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:01,618 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 00:55:01,618 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 00:55:01,619 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,619 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,620 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:01,620 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:01,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:01,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:01,620 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:01,620 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:01,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,621 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:01,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,621 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,621 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 00:55:01,621 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:01,621 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 00:55:01,621 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:01,621 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 00:55:01,621 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:55:01,621 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:01,621 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:01,621 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. started... 2023-02-13 00:55:01,622 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing cfb81dcb75adebd110154b74cfdaab0a 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:55:02,034 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201bb83d99ae33049a18c8205e97d02e104 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201bb83d99ae33049a18c8205e97d02e104 2023-02-13 00:55:02,440 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/.tmp/family1/2352b3a46e2f45ab8d31dc1ce66ab924 2023-02-13 00:55:02,445 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/.tmp/family1/2352b3a46e2f45ab8d31dc1ce66ab924 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/2352b3a46e2f45ab8d31dc1ce66ab924 2023-02-13 00:55:02,449 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/2352b3a46e2f45ab8d31dc1ce66ab924, entries=1, sequenceid=13, filesize=5.0 K 2023-02-13 00:55:02,449 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cfb81dcb75adebd110154b74cfdaab0a in 827ms, sequenceid=13, compaction requested=true 2023-02-13 00:55:02,449 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for cfb81dcb75adebd110154b74cfdaab0a: 2023-02-13 00:55:02,449 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:02,450 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:55:02,450 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:55:02,450 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:02,450 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 00:55:02,450 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 00:55:02,451 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:02,451 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:02,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:02,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:02,452 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,452 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:02,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:02,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:02,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:02,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:02,454 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 00:55:02,454 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@670e025a[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:02,454 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 00:55:02,455 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,457 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,457 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 00:55:02,457 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 00:55:02,457 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:55:02,457 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:02,457 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:02,458 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 00:55:02,458 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:02,458 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:02,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:02,458 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 00:55:02,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:02,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:02,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:02,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:02,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:02,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:02,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:02,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:55:02,461 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 00:55:02,461 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 00:55:02,461 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:02,461 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:02,461 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,461 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:02,462 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,462 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,462 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:02,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:02,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:02,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:02,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:02,463 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:02,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:02,464 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:02,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:02,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:02,466 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:02,466 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:02,466 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:02,466 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 00:55:02,466 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:02,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:02,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:02,466 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:02,466 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,466 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,466 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:02,467 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,467 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,467 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,467 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 00:55:02,467 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:02,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:02,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:02,567 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:02,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:02,571 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:02,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 00:55:02,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,573 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 00:55:02,573 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:02,573 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 00:55:02,573 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 00:55:02,573 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,573 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,575 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:02,575 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:02,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:02,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:02,576 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:02,576 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:02,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,576 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:02,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,576 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,577 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 00:55:02,577 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:02,577 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 00:55:02,577 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:02,577 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 00:55:02,577 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:55:02,577 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:02,577 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:02,577 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. started... 2023-02-13 00:55:02,577 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing cfb81dcb75adebd110154b74cfdaab0a 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:55:02,991 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512050e42340d7d0c4e088ef11674567aa9a1 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512050e42340d7d0c4e088ef11674567aa9a1 2023-02-13 00:55:03,397 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/.tmp/family1/8f991f759ebb45988106c8506ff099be 2023-02-13 00:55:03,402 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/.tmp/family1/8f991f759ebb45988106c8506ff099be as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/8f991f759ebb45988106c8506ff099be 2023-02-13 00:55:03,407 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/8f991f759ebb45988106c8506ff099be, entries=1, sequenceid=17, filesize=5.0 K 2023-02-13 00:55:03,407 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cfb81dcb75adebd110154b74cfdaab0a in 830ms, sequenceid=17, compaction requested=true 2023-02-13 00:55:03,408 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for cfb81dcb75adebd110154b74cfdaab0a: 2023-02-13 00:55:03,408 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:03,408 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:55:03,408 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:55:03,408 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:03,408 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 00:55:03,408 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 00:55:03,410 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:03,410 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,410 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,411 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:03,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:03,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:03,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:03,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:03,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:03,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:03,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:03,412 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 00:55:03,412 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e50e85b[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:03,412 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 00:55:03,412 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,416 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,416 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 00:55:03,416 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 00:55:03,416 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:03,416 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:03,416 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:55:03,419 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 00:55:03,419 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:03,419 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:03,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:03,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:03,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:03,419 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 00:55:03,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:03,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:03,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:03,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:03,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:03,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:55:03,421 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 00:55:03,421 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 00:55:03,421 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:03,421 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:03,421 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,421 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:03,423 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,423 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:03,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:03,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,423 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:03,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:03,424 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:03,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:03,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:03,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,426 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:03,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:03,428 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:03,428 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,428 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,429 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,428 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,429 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:03,429 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,429 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,429 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 00:55:03,429 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:03,429 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:03,429 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,429 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 00:55:03,429 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:03,530 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:03,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:03,534 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:03,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 00:55:03,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,535 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 00:55:03,535 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:03,535 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 00:55:03,535 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 00:55:03,536 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,536 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,537 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:03,537 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,537 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:03,537 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:03,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,538 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:03,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,538 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,538 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 00:55:03,539 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:03,539 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 00:55:03,539 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:03,539 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 00:55:03,539 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:55:03,539 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:03,539 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:03,539 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. started... 2023-02-13 00:55:03,539 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing cfb81dcb75adebd110154b74cfdaab0a 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:55:03,952 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228f38b0abc57744126a7c11d33e936d6d4 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228f38b0abc57744126a7c11d33e936d6d4 2023-02-13 00:55:04,362 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/.tmp/family1/178bba9e123c47778a7dc2880bab141e 2023-02-13 00:55:04,366 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/.tmp/family1/178bba9e123c47778a7dc2880bab141e as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/178bba9e123c47778a7dc2880bab141e 2023-02-13 00:55:04,371 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/178bba9e123c47778a7dc2880bab141e, entries=1, sequenceid=21, filesize=5.0 K 2023-02-13 00:55:04,372 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cfb81dcb75adebd110154b74cfdaab0a in 833ms, sequenceid=21, compaction requested=true 2023-02-13 00:55:04,372 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for cfb81dcb75adebd110154b74cfdaab0a: 2023-02-13 00:55:04,372 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:04,372 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:55:04,372 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:55:04,372 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,372 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 00:55:04,372 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 00:55:04,374 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,374 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:04,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:04,374 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,374 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:04,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:04,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:04,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:04,375 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 00:55:04,375 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5c7ca711[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:04,375 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 00:55:04,375 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,376 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,376 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 00:55:04,376 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 00:55:04,376 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:55:04,376 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,377 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:04,379 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,379 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 00:55:04,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:04,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:04,379 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:04,379 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 00:55:04,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:04,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:04,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:04,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:55:04,381 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 00:55:04,381 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 00:55:04,381 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:04,381 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:04,381 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,381 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:04,382 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,382 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,382 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:04,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:04,382 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:04,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:04,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:04,383 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:04,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:04,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,384 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:04,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,386 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,386 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:04,386 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:04,386 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 00:55:04,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:04,386 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,386 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:04,386 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:04,386 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,386 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:04,386 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:04,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:04,386 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,386 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,387 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 00:55:04,387 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:04,487 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:04,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:04,491 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:04,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 00:55:04,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,493 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 00:55:04,493 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:04,493 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 00:55:04,493 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 00:55:04,493 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,494 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,495 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,495 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:04,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:04,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:04,495 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,495 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:04,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,496 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:04,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,496 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,496 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 00:55:04,496 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,496 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 00:55:04,496 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:04,496 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 00:55:04,496 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:55:04,497 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:04,497 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:04,497 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. started... 2023-02-13 00:55:04,497 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing cfb81dcb75adebd110154b74cfdaab0a 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:55:04,508 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512313fb29c213ce04706a2df5e1535f1121f to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512313fb29c213ce04706a2df5e1535f1121f 2023-02-13 00:55:04,913 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/.tmp/family1/30aff31171de4ae5ad1c189ba98f9a1c 2023-02-13 00:55:04,918 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/.tmp/family1/30aff31171de4ae5ad1c189ba98f9a1c as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/30aff31171de4ae5ad1c189ba98f9a1c 2023-02-13 00:55:04,923 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/30aff31171de4ae5ad1c189ba98f9a1c, entries=1, sequenceid=25, filesize=5.0 K 2023-02-13 00:55:04,924 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cfb81dcb75adebd110154b74cfdaab0a in 427ms, sequenceid=25, compaction requested=true 2023-02-13 00:55:04,924 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for cfb81dcb75adebd110154b74cfdaab0a: 2023-02-13 00:55:04,924 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:04,924 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:55:04,924 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:55:04,924 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,924 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 00:55:04,924 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 00:55:04,927 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,927 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:04,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:04,927 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,927 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:04,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:04,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:04,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:04,929 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 00:55:04,929 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6bae2c54[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:04,929 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 00:55:04,929 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,930 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,930 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 00:55:04,930 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 00:55:04,931 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,931 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:04,931 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:55:04,932 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 00:55:04,932 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:04,932 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,932 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 00:55:04,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:04,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:04,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:04,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:04,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:04,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:55:04,934 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 00:55:04,934 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 00:55:04,935 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:04,935 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:04,935 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,935 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:04,936 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,936 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:04,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:04,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,936 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:04,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:04,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:04,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:04,937 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:04,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,938 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:04,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,945 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,945 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,945 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,945 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:04,945 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,945 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,945 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:04,945 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:04,945 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 00:55:04,945 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:04,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:04,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:04,945 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:04,945 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:04,945 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-13 00:55:04,945 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 00:55:04,945 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:04,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:04,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:05,045 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:05,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:05,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:05,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 00:55:05,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,052 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 00:55:05,052 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:05,053 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 00:55:05,053 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 00:55:05,053 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,053 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,058 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:05,058 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:05,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:05,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:05,058 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:05,058 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:05,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,059 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:05,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,059 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,060 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 00:55:05,060 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,060 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 00:55:05,060 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:05,060 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 00:55:05,060 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:55:05,060 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:05,060 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:05,060 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. started... 2023-02-13 00:55:05,061 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing cfb81dcb75adebd110154b74cfdaab0a 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:55:05,474 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601017a2319ab7b634e9ebb8d7c33fb060d3d to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601017a2319ab7b634e9ebb8d7c33fb060d3d 2023-02-13 00:55:05,882 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/.tmp/family1/ec9b063b5e07460d85f0b8b4ea0002c4 2023-02-13 00:55:05,886 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/.tmp/family1/ec9b063b5e07460d85f0b8b4ea0002c4 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/ec9b063b5e07460d85f0b8b4ea0002c4 2023-02-13 00:55:05,890 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/ec9b063b5e07460d85f0b8b4ea0002c4, entries=1, sequenceid=29, filesize=5.0 K 2023-02-13 00:55:05,891 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cfb81dcb75adebd110154b74cfdaab0a in 830ms, sequenceid=29, compaction requested=true 2023-02-13 00:55:05,891 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for cfb81dcb75adebd110154b74cfdaab0a: 2023-02-13 00:55:05,891 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:05,891 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:55:05,891 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:55:05,891 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:05,891 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 00:55:05,891 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 00:55:05,893 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,893 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:05,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:05,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:05,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:05,893 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,893 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:05,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:05,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:05,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:05,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:05,894 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 00:55:05,894 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7c25efa9[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:05,894 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 00:55:05,894 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,895 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,896 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 00:55:05,896 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 00:55:05,896 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:55:05,896 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:05,896 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:05,898 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:05,898 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 00:55:05,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:05,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:05,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:05,898 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:05,898 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 00:55:05,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:05,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:05,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:05,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:05,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:05,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:55:05,900 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 00:55:05,900 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 00:55:05,900 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:05,900 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:05,900 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,900 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:05,901 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,901 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:05,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:05,902 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:05,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:05,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:05,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:05,902 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:05,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:05,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,903 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:05,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:05,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:05,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:05,905 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:05,905 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:05,905 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,905 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 00:55:05,906 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:05,905 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,905 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:05,906 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:05,906 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-13 00:55:05,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:05,906 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 00:55:05,905 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:05,906 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:05,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:05,906 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,906 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,906 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:05,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:05,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:06,006 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:06,007 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:06,011 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:06,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 00:55:06,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,013 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 00:55:06,013 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:06,013 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 00:55:06,013 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 00:55:06,014 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,014 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,015 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:06,015 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:06,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:06,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:06,015 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:06,015 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:06,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,016 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:06,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,016 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,016 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 00:55:06,016 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,016 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 00:55:06,016 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:06,016 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 00:55:06,020 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:55:06,020 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:06,020 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:06,020 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. started... 2023-02-13 00:55:06,020 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing cfb81dcb75adebd110154b74cfdaab0a 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:55:06,037 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103d10be6ab094b40fbab26978b69701101 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103d10be6ab094b40fbab26978b69701101 2023-02-13 00:55:06,442 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/.tmp/family1/501269bc1f89416ab633e9ebd700aa43 2023-02-13 00:55:06,447 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/.tmp/family1/501269bc1f89416ab633e9ebd700aa43 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/501269bc1f89416ab633e9ebd700aa43 2023-02-13 00:55:06,450 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/501269bc1f89416ab633e9ebd700aa43, entries=1, sequenceid=33, filesize=5.0 K 2023-02-13 00:55:06,451 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cfb81dcb75adebd110154b74cfdaab0a in 431ms, sequenceid=33, compaction requested=true 2023-02-13 00:55:06,451 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for cfb81dcb75adebd110154b74cfdaab0a: 2023-02-13 00:55:06,451 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:06,451 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:55:06,451 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:55:06,451 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:06,451 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 00:55:06,451 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 00:55:06,454 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,454 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:06,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:06,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:06,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:06,454 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,454 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:06,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:06,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:06,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:06,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:06,455 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 00:55:06,455 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@450ed246[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:06,455 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 00:55:06,455 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,456 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,456 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 00:55:06,457 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 00:55:06,457 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:55:06,457 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:06,457 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:06,458 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:06,458 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 00:55:06,458 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:06,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:06,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:06,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:06,458 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 00:55:06,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:06,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:06,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:06,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:06,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:06,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:55:06,460 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 00:55:06,460 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 00:55:06,460 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:06,460 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:06,460 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,460 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:06,461 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,461 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:06,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:06,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,461 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:06,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:06,462 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:06,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:06,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:06,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:06,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,463 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:06,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:06,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:06,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:06,466 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:06,466 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,466 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:06,466 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:06,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:06,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:06,466 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,467 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-13 00:55:06,467 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:06,467 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:06,466 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 00:55:06,467 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:06,467 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:06,467 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:06,467 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,467 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 00:55:06,467 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:06,567 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:06,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:06,572 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:06,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 00:55:06,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,574 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 00:55:06,574 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:06,574 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 00:55:06,574 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 00:55:06,575 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,575 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,576 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:06,576 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:06,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:06,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:06,577 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:06,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,577 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:06,577 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:06,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,578 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,578 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 00:55:06,578 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:06,578 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 00:55:06,578 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:06,578 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 00:55:06,578 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:55:06,578 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:06,578 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:06,578 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. started... 2023-02-13 00:55:06,578 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing cfb81dcb75adebd110154b74cfdaab0a 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:55:06,605 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907518ce99176ce4d61ad4055e8c156245b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20150907518ce99176ce4d61ad4055e8c156245b 2023-02-13 00:55:07,011 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/.tmp/family1/c6544bfce4f74b188cd9e205310c1802 2023-02-13 00:55:07,015 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/.tmp/family1/c6544bfce4f74b188cd9e205310c1802 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/c6544bfce4f74b188cd9e205310c1802 2023-02-13 00:55:07,019 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/c6544bfce4f74b188cd9e205310c1802, entries=1, sequenceid=37, filesize=5.0 K 2023-02-13 00:55:07,020 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cfb81dcb75adebd110154b74cfdaab0a in 442ms, sequenceid=37, compaction requested=true 2023-02-13 00:55:07,020 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for cfb81dcb75adebd110154b74cfdaab0a: 2023-02-13 00:55:07,020 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:07,020 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:55:07,020 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:55:07,020 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,020 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 00:55:07,020 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 00:55:07,022 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,022 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:07,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:07,022 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,022 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:07,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:07,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:07,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:07,023 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 00:55:07,024 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7cf3a1d9[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:07,024 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 00:55:07,024 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,026 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,026 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 00:55:07,026 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 00:55:07,026 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:55:07,026 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,026 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:07,028 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 00:55:07,028 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:07,028 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,028 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 00:55:07,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:07,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:07,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:07,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:07,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:07,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:55:07,030 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 00:55:07,030 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 00:55:07,030 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:07,030 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:07,030 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,030 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:07,032 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,032 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:07,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:07,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,032 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:07,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:07,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:07,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:07,032 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:07,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,034 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:07,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,036 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,036 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,036 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:07,036 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,036 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:07,036 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,036 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 00:55:07,036 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:07,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:07,036 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:07,036 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,036 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:07,036 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:07,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:07,036 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 00:55:07,036 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:07,137 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:07,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:07,141 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:07,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 00:55:07,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,143 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 00:55:07,143 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:07,143 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 00:55:07,143 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 00:55:07,143 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,143 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,146 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,146 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:07,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:07,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:07,146 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,146 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:07,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,146 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:07,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,147 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,147 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 00:55:07,147 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,147 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 00:55:07,147 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:07,147 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 00:55:07,147 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:55:07,147 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:07,147 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:07,147 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. started... 2023-02-13 00:55:07,147 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing cfb81dcb75adebd110154b74cfdaab0a 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:55:07,559 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511200f319043c365459dbaeb7588ae4fa35c to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511200f319043c365459dbaeb7588ae4fa35c 2023-02-13 00:55:07,964 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/.tmp/family1/e0d3a137847246a482867db6c0fc7843 2023-02-13 00:55:07,968 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/.tmp/family1/e0d3a137847246a482867db6c0fc7843 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/e0d3a137847246a482867db6c0fc7843 2023-02-13 00:55:07,972 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/e0d3a137847246a482867db6c0fc7843, entries=1, sequenceid=41, filesize=5.0 K 2023-02-13 00:55:07,973 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cfb81dcb75adebd110154b74cfdaab0a in 826ms, sequenceid=41, compaction requested=true 2023-02-13 00:55:07,973 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for cfb81dcb75adebd110154b74cfdaab0a: 2023-02-13 00:55:07,973 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:07,973 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:55:07,973 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:55:07,973 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,973 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 00:55:07,973 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 00:55:07,976 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,976 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:07,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:07,976 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,976 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:07,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:07,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:07,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:07,977 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 00:55:07,977 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6625ee9f[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:07,977 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 00:55:07,977 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,979 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,979 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 00:55:07,979 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 00:55:07,979 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,979 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:07,979 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:55:07,981 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,981 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 00:55:07,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,981 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:07,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:07,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:07,981 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 00:55:07,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:07,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:07,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:07,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:55:07,983 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 00:55:07,983 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 00:55:07,983 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:07,983 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:07,983 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,983 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:07,984 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,984 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:07,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:07,984 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:07,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:07,985 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:07,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:07,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:07,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,986 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:07,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,989 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,989 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:07,989 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:07,989 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 00:55:07,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:07,989 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:07,989 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:07,989 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,989 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:07,989 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-13 00:55:07,989 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:07,989 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 00:55:07,989 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,989 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:07,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:07,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:07,989 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:07,990 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:08,089 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:08,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:08,091 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43563] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:08,093 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] procedure2.ProcedureExecutor(1029): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 00:55:08,093 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 00:55:08,094 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:55:08,094 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676250308094 2023-02-13 00:55:08,095 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 00:55:08,096 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-13 00:55:08,096 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 00:55:08,096 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] zookeeper.ReadOnlyZKClient(139): Connect 0x0c2c4808 to 127.0.0.1:54478 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:55:08,100 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4e7448e9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:55:08,101 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:08,934 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103c38b4916451543a6a6f93abee1f414ef to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103c38b4916451543a6a6f93abee1f414ef 2023-02-13 00:55:08,934 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 00:55:08,947 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:55:08,948 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49826, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:55:08,956 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 00:55:08,957 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48048, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 00:55:08,965 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dee15c6c3df343cba3cb01527bc33831 first=Optional[r3] last=Optional[r6] 2023-02-13 00:55:08,965 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dee15c6c3df343cba3cb01527bc33831}] 2023-02-13 00:55:08,970 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dee15c6c3df343cba3cb01527bc33831 for inclusion in cfb81dcb75adebd110154b74cfdaab0a/family1 2023-02-13 00:55:08,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-13 00:55:08,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 00:55:08,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for cfb81dcb75adebd110154b74cfdaab0a: 2023-02-13 00:55:08,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dee15c6c3df343cba3cb01527bc33831 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__gcmsdkhhurouj8mo33otuufdphbutrvjtsi4jgtio2t6laf8198buorpijg7ct9g/family1/dee15c6c3df343cba3cb01527bc33831 2023-02-13 00:55:08,975 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__gcmsdkhhurouj8mo33otuufdphbutrvjtsi4jgtio2t6laf8198buorpijg7ct9g/family1/dee15c6c3df343cba3cb01527bc33831 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/4bceb316d67d4aa18627dcace82d541c_SeqId_44_ 2023-02-13 00:55:08,976 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__gcmsdkhhurouj8mo33otuufdphbutrvjtsi4jgtio2t6laf8198buorpijg7ct9g/family1/dee15c6c3df343cba3cb01527bc33831 into cfb81dcb75adebd110154b74cfdaab0a/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/4bceb316d67d4aa18627dcace82d541c_SeqId_44_ - updating store file list. 2023-02-13 00:55:08,979 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/4bceb316d67d4aa18627dcace82d541c_SeqId_44_ into cfb81dcb75adebd110154b74cfdaab0a/family1 2023-02-13 00:55:08,980 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__gcmsdkhhurouj8mo33otuufdphbutrvjtsi4jgtio2t6laf8198buorpijg7ct9g/family1/dee15c6c3df343cba3cb01527bc33831 into cfb81dcb75adebd110154b74cfdaab0a/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/4bceb316d67d4aa18627dcace82d541c_SeqId_44_) 2023-02-13 00:55:08,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__gcmsdkhhurouj8mo33otuufdphbutrvjtsi4jgtio2t6laf8198buorpijg7ct9g/family1/dee15c6c3df343cba3cb01527bc33831 2023-02-13 00:55:08,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-13 00:55:08,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-13 00:55:08,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(1912): cfb81dcb75adebd110154b74cfdaab0a/family1 is initiating minor compaction 2023-02-13 00:55:08,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4281c592; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:55:08,983 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of cfb81dcb75adebd110154b74cfdaab0a/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:08,983 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/87ab412cb9694811843cfeaa4b1dd9a9, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/2352b3a46e2f45ab8d31dc1ce66ab924, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/8f991f759ebb45988106c8506ff099be, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/178bba9e123c47778a7dc2880bab141e, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/30aff31171de4ae5ad1c189ba98f9a1c, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/ec9b063b5e07460d85f0b8b4ea0002c4, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/501269bc1f89416ab633e9ebd700aa43, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/c6544bfce4f74b188cd9e205310c1802, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/e0d3a137847246a482867db6c0fc7843, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/4bceb316d67d4aa18627dcace82d541c_SeqId_44_] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/.tmp, totalSize=50.3 K 2023-02-13 00:55:08,984 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 87ab412cb9694811843cfeaa4b1dd9a9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-13 00:55:08,984 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 2352b3a46e2f45ab8d31dc1ce66ab924, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-13 00:55:08,984 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 8f991f759ebb45988106c8506ff099be, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-13 00:55:08,984 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 178bba9e123c47778a7dc2880bab141e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-13 00:55:08,984 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 30aff31171de4ae5ad1c189ba98f9a1c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-13 00:55:08,984 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting ec9b063b5e07460d85f0b8b4ea0002c4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-13 00:55:08,984 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 501269bc1f89416ab633e9ebd700aa43, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-13 00:55:08,984 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting c6544bfce4f74b188cd9e205310c1802, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-13 00:55:08,984 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting e0d3a137847246a482867db6c0fc7843, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-13 00:55:08,984 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 4bceb316d67d4aa18627dcace82d541c_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-13 00:55:08,985 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 00:55:08,986 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:08,987 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228f38b0abc57744126a7c11d33e936d6d4 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228f38b0abc57744126a7c11d33e936d6d4 2023-02-13 00:55:08,989 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512313fb29c213ce04706a2df5e1535f1121f to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512313fb29c213ce04706a2df5e1535f1121f 2023-02-13 00:55:08,991 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601017a2319ab7b634e9ebb8d7c33fb060d3d to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601017a2319ab7b634e9ebb8d7c33fb060d3d 2023-02-13 00:55:08,992 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103d10be6ab094b40fbab26978b69701101 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103d10be6ab094b40fbab26978b69701101 2023-02-13 00:55:08,992 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-02-13 00:55:08,992 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-13 00:55:08,992 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:08,992 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:08,993 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:08,993 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:08,993 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:08,993 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:08,993 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:08,993 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:09,020 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] throttle.PressureAwareThroughputController(145): cfb81dcb75adebd110154b74cfdaab0a#family1#compaction#72 average throughput is 0.20 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:55:09,442 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205aea58276b27c4023ada598df5d33e2aa to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205aea58276b27c4023ada598df5d33e2aa 2023-02-13 00:55:09,443 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/.tmp/family1/55f879dce5084f20b29f673857f69a0d as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/55f879dce5084f20b29f673857f69a0d 2023-02-13 00:55:09,443 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 00:55:09,447 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in cfb81dcb75adebd110154b74cfdaab0a/family1 of cfb81dcb75adebd110154b74cfdaab0a into 55f879dce5084f20b29f673857f69a0d(size=6.8 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:55:09,447 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for cfb81dcb75adebd110154b74cfdaab0a: 2023-02-13 00:55:09,447 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1676249708983; duration=0sec 2023-02-13 00:55:09,448 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:55:09,466 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/d51cc456701e461195269c7d90d0b468 first=Optional[r00] last=Optional[r2] 2023-02-13 00:55:09,466 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/d51cc456701e461195269c7d90d0b468}] 2023-02-13 00:55:09,468 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/d51cc456701e461195269c7d90d0b468 for inclusion in cfb81dcb75adebd110154b74cfdaab0a/family1 2023-02-13 00:55:09,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-13 00:55:09,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 00:55:09,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for cfb81dcb75adebd110154b74cfdaab0a: 2023-02-13 00:55:09,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/d51cc456701e461195269c7d90d0b468 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__mfeklrkiv56ojoitg2cf75tk3v7243fpacgm1fio1mm57a0fo2lfuk8364d67t0d/family1/d51cc456701e461195269c7d90d0b468 2023-02-13 00:55:09,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__mfeklrkiv56ojoitg2cf75tk3v7243fpacgm1fio1mm57a0fo2lfuk8364d67t0d/family1/d51cc456701e461195269c7d90d0b468 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/18d0e75f0b3641638afec94be590cf00_SeqId_47_ 2023-02-13 00:55:09,475 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__mfeklrkiv56ojoitg2cf75tk3v7243fpacgm1fio1mm57a0fo2lfuk8364d67t0d/family1/d51cc456701e461195269c7d90d0b468 into cfb81dcb75adebd110154b74cfdaab0a/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/18d0e75f0b3641638afec94be590cf00_SeqId_47_ - updating store file list. 2023-02-13 00:55:09,477 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/18d0e75f0b3641638afec94be590cf00_SeqId_47_ into cfb81dcb75adebd110154b74cfdaab0a/family1 2023-02-13 00:55:09,477 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__mfeklrkiv56ojoitg2cf75tk3v7243fpacgm1fio1mm57a0fo2lfuk8364d67t0d/family1/d51cc456701e461195269c7d90d0b468 into cfb81dcb75adebd110154b74cfdaab0a/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/18d0e75f0b3641638afec94be590cf00_SeqId_47_) 2023-02-13 00:55:09,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__mfeklrkiv56ojoitg2cf75tk3v7243fpacgm1fio1mm57a0fo2lfuk8364d67t0d/family1/d51cc456701e461195269c7d90d0b468 2023-02-13 00:55:09,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-13 00:55:09,479 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-13 00:55:09,479 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(1912): cfb81dcb75adebd110154b74cfdaab0a/family1 is initiating minor compaction (all files) 2023-02-13 00:55:09,479 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4d1c82dc; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:55:09,479 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of cfb81dcb75adebd110154b74cfdaab0a/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:09,479 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/d481003d34da46c388519b8457624e6c, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/55f879dce5084f20b29f673857f69a0d, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/18d0e75f0b3641638afec94be590cf00_SeqId_47_] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/.tmp, totalSize=388.1 K 2023-02-13 00:55:09,479 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting d481003d34da46c388519b8457624e6c, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878845884 2023-02-13 00:55:09,480 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 55f879dce5084f20b29f673857f69a0d, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621245884 2023-02-13 00:55:09,480 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 18d0e75f0b3641638afec94be590cf00_SeqId_47_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878845884 2023-02-13 00:55:09,480 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 00:55:09,481 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:09,482 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130cdc47d88ec8c44eb9471152e592e8566 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130cdc47d88ec8c44eb9471152e592e8566 2023-02-13 00:55:09,483 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201bb83d99ae33049a18c8205e97d02e104 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201bb83d99ae33049a18c8205e97d02e104 2023-02-13 00:55:09,485 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512050e42340d7d0c4e088ef11674567aa9a1 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512050e42340d7d0c4e088ef11674567aa9a1 2023-02-13 00:55:09,485 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-02-13 00:55:09,485 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-13 00:55:09,485 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:09,485 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 00:55:09,485 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:55:09,486 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 00:55:09,487 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.3940 sec 2023-02-13 00:55:09,498 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] throttle.PressureAwareThroughputController(145): cfb81dcb75adebd110154b74cfdaab0a#family1#compaction#73 average throughput is 17.96 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:55:09,561 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/87ab412cb9694811843cfeaa4b1dd9a9, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/2352b3a46e2f45ab8d31dc1ce66ab924, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/8f991f759ebb45988106c8506ff099be, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/178bba9e123c47778a7dc2880bab141e, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/30aff31171de4ae5ad1c189ba98f9a1c, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/ec9b063b5e07460d85f0b8b4ea0002c4, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/501269bc1f89416ab633e9ebd700aa43, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/c6544bfce4f74b188cd9e205310c1802, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/e0d3a137847246a482867db6c0fc7843, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/4bceb316d67d4aa18627dcace82d541c_SeqId_44_] to archive 2023-02-13 00:55:09,562 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-13 00:55:09,562 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:09,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure2.ProcedureExecutor(1029): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:09,565 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/87ab412cb9694811843cfeaa4b1dd9a9 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/87ab412cb9694811843cfeaa4b1dd9a9 2023-02-13 00:55:09,566 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/2352b3a46e2f45ab8d31dc1ce66ab924 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/2352b3a46e2f45ab8d31dc1ce66ab924 2023-02-13 00:55:09,567 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/8f991f759ebb45988106c8506ff099be to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/8f991f759ebb45988106c8506ff099be 2023-02-13 00:55:09,568 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/178bba9e123c47778a7dc2880bab141e to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/178bba9e123c47778a7dc2880bab141e 2023-02-13 00:55:09,570 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/30aff31171de4ae5ad1c189ba98f9a1c to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/30aff31171de4ae5ad1c189ba98f9a1c 2023-02-13 00:55:09,571 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/ec9b063b5e07460d85f0b8b4ea0002c4 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/ec9b063b5e07460d85f0b8b4ea0002c4 2023-02-13 00:55:09,572 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/501269bc1f89416ab633e9ebd700aa43 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/501269bc1f89416ab633e9ebd700aa43 2023-02-13 00:55:09,573 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/c6544bfce4f74b188cd9e205310c1802 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/c6544bfce4f74b188cd9e205310c1802 2023-02-13 00:55:09,574 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/e0d3a137847246a482867db6c0fc7843 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/e0d3a137847246a482867db6c0fc7843 2023-02-13 00:55:09,575 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/4bceb316d67d4aa18627dcace82d541c_SeqId_44_ to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/4bceb316d67d4aa18627dcace82d541c_SeqId_44_ 2023-02-13 00:55:09,982 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-13 00:55:09,982 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-02-13 00:55:09,983 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-13 00:55:09,985 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-02-13 00:55:09,987 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=cfb81dcb75adebd110154b74cfdaab0a, REOPEN/MOVE}] 2023-02-13 00:55:09,987 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=cfb81dcb75adebd110154b74cfdaab0a, REOPEN/MOVE 2023-02-13 00:55:09,988 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=cfb81dcb75adebd110154b74cfdaab0a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:09,988 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676249709988"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249709988"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249709988"}]},"ts":"1676249709988"} 2023-02-13 00:55:09,989 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure cfb81dcb75adebd110154b74cfdaab0a, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:55:10,141 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:55:10,141 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing cfb81dcb75adebd110154b74cfdaab0a, disabling compactions & flushes 2023-02-13 00:55:10,141 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:10,716 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/.tmp/family1/ab0387741cb24f188ca49511d9a919c6 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/ab0387741cb24f188ca49511d9a919c6 2023-02-13 00:55:10,720 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in cfb81dcb75adebd110154b74cfdaab0a/family1 of cfb81dcb75adebd110154b74cfdaab0a into ab0387741cb24f188ca49511d9a919c6(size=192.0 K), total size for store is 192.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 00:55:10,721 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for cfb81dcb75adebd110154b74cfdaab0a: 2023-02-13 00:55:10,721 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:10,721 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a., storeName=family1, fileCount=3, fileSize=388.1 K (190.5 K, 6.8 K, 190.9 K), priority=2, time=1676249709479; duration=1sec 2023-02-13 00:55:10,721 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:10,721 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. after waiting 0 ms 2023-02-13 00:55:10,721 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:55:10,721 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:10,724 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/d481003d34da46c388519b8457624e6c, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/55f879dce5084f20b29f673857f69a0d, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/18d0e75f0b3641638afec94be590cf00_SeqId_47_] to archive 2023-02-13 00:55:10,724 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:10,726 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/d481003d34da46c388519b8457624e6c to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/d481003d34da46c388519b8457624e6c 2023-02-13 00:55:10,727 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/55f879dce5084f20b29f673857f69a0d to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/55f879dce5084f20b29f673857f69a0d 2023-02-13 00:55:10,728 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/18d0e75f0b3641638afec94be590cf00_SeqId_47_ to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/18d0e75f0b3641638afec94be590cf00_SeqId_47_ 2023-02-13 00:55:10,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-13 00:55:10,732 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:10,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for cfb81dcb75adebd110154b74cfdaab0a: Waiting for close lock at 1676249710141Running coprocessor pre-close hooks at 1676249710141Disabling compacts and flushes for region at 1676249710141Disabling writes for close at 1676249710721 (+580 ms)Writing region close event to WAL at 1676249710729 (+8 ms)Running coprocessor post-close hooks at 1676249710732 (+3 ms)Closed at 1676249710732 2023-02-13 00:55:10,732 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: cfb81dcb75adebd110154b74cfdaab0a to self. 2023-02-13 00:55:10,734 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:55:10,734 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=cfb81dcb75adebd110154b74cfdaab0a, regionState=CLOSED 2023-02-13 00:55:10,734 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676249710734"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249710734"}]},"ts":"1676249710734"} 2023-02-13 00:55:10,736 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=61 2023-02-13 00:55:10,736 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure cfb81dcb75adebd110154b74cfdaab0a, server=jenkins-hbase4.apache.org,36605,1676249622318 in 746 msec 2023-02-13 00:55:10,737 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=cfb81dcb75adebd110154b74cfdaab0a, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=true 2023-02-13 00:55:10,887 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=cfb81dcb75adebd110154b74cfdaab0a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:10,887 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676249710887"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249710887"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249710887"}]},"ts":"1676249710887"} 2023-02-13 00:55:10,889 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure cfb81dcb75adebd110154b74cfdaab0a, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:55:11,043 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:11,043 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => cfb81dcb75adebd110154b74cfdaab0a, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:55:11,043 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:55:11,043 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:11,043 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:55:11,043 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:55:11,044 INFO [StoreOpener-cfb81dcb75adebd110154b74cfdaab0a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:55:11,045 DEBUG [StoreOpener-cfb81dcb75adebd110154b74cfdaab0a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1 2023-02-13 00:55:11,045 DEBUG [StoreOpener-cfb81dcb75adebd110154b74cfdaab0a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1 2023-02-13 00:55:11,046 INFO [StoreOpener-cfb81dcb75adebd110154b74cfdaab0a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cfb81dcb75adebd110154b74cfdaab0a columnFamilyName family1 2023-02-13 00:55:11,050 DEBUG [StoreOpener-cfb81dcb75adebd110154b74cfdaab0a-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/ab0387741cb24f188ca49511d9a919c6 2023-02-13 00:55:11,050 INFO [StoreOpener-cfb81dcb75adebd110154b74cfdaab0a-1] regionserver.HStore(310): Store=cfb81dcb75adebd110154b74cfdaab0a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:11,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:55:11,052 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:55:11,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:55:11,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened cfb81dcb75adebd110154b74cfdaab0a; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71017752, jitterRate=0.05824697017669678}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 00:55:11,055 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for cfb81dcb75adebd110154b74cfdaab0a: Running coprocessor pre-open hook at 1676249711044Writing region info on filesystem at 1676249711044Initializing all the Stores at 1676249711044Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1676249711044Cleaning up temporary data from old regions at 1676249711052 (+8 ms)Cleaning up detritus from prior splits at 1676249711053 (+1 ms)Running coprocessor post-open hooks at 1676249711055 (+2 ms)Region opened successfully at 1676249711055 2023-02-13 00:55:11,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a., pid=63, masterSystemTime=1676249711040 2023-02-13 00:55:11,057 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:11,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:11,057 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=cfb81dcb75adebd110154b74cfdaab0a, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:11,057 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676249711057"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249711057"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249711057"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249711057"}]},"ts":"1676249711057"} 2023-02-13 00:55:11,059 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=63, resume processing ppid=61 2023-02-13 00:55:11,059 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure cfb81dcb75adebd110154b74cfdaab0a, server=jenkins-hbase4.apache.org,36605,1676249622318 in 169 msec 2023-02-13 00:55:11,060 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-02-13 00:55:11,061 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=cfb81dcb75adebd110154b74cfdaab0a, REOPEN/MOVE in 1.0720 sec 2023-02-13 00:55:11,062 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=59 2023-02-13 00:55:11,062 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.0760 sec 2023-02-13 00:55:11,064 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.5010 sec 2023-02-13 00:55:11,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=59 2023-02-13 00:55:11,066 INFO [Listener at localhost/37869] compactions.TestMobCompactor(278): alter status finished 2023-02-13 00:55:11,066 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43563] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:11,068 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] procedure2.ProcedureExecutor(1029): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 00:55:11,068 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 00:55:11,069 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:55:11,069 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676250311069 2023-02-13 00:55:11,069 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 00:55:11,070 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-13 00:55:11,070 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 00:55:11,070 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] zookeeper.ReadOnlyZKClient(139): Connect 0x4b182120 to 127.0.0.1:54478 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:55:11,075 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6866cf65, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:55:11,075 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:11,075 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:11,075 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:11,075 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:11,075 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:11,899 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128e2cc6dfba34d4ebf81b623277d665383 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128e2cc6dfba34d4ebf81b623277d665383 2023-02-13 00:55:11,900 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 00:55:11,912 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:55:11,913 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49836, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:55:11,919 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 00:55:11,920 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48056, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 00:55:11,928 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/be354e1570584b5386ed15a9afc375a1 first=Optional[r05] last=Optional[r8] 2023-02-13 00:55:11,929 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=51 for row with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/be354e1570584b5386ed15a9afc375a1}] 2023-02-13 00:55:11,933 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/be354e1570584b5386ed15a9afc375a1 for inclusion in cfb81dcb75adebd110154b74cfdaab0a/family1 2023-02-13 00:55:11,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-13 00:55:11,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 00:55:11,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for cfb81dcb75adebd110154b74cfdaab0a: 2023-02-13 00:55:11,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/be354e1570584b5386ed15a9afc375a1 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__7rgsq9a0vr2e6ivp4n42hiip69c7re11emhk694hgf8mbvjmdj6lha4iqqm4c54s/family1/be354e1570584b5386ed15a9afc375a1 2023-02-13 00:55:11,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__7rgsq9a0vr2e6ivp4n42hiip69c7re11emhk694hgf8mbvjmdj6lha4iqqm4c54s/family1/be354e1570584b5386ed15a9afc375a1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/d0caf53bc40b4e45a0bff39fc9e0f77b_SeqId_53_ 2023-02-13 00:55:11,938 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__7rgsq9a0vr2e6ivp4n42hiip69c7re11emhk694hgf8mbvjmdj6lha4iqqm4c54s/family1/be354e1570584b5386ed15a9afc375a1 into cfb81dcb75adebd110154b74cfdaab0a/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/d0caf53bc40b4e45a0bff39fc9e0f77b_SeqId_53_ - updating store file list. 2023-02-13 00:55:11,941 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/d0caf53bc40b4e45a0bff39fc9e0f77b_SeqId_53_ into cfb81dcb75adebd110154b74cfdaab0a/family1 2023-02-13 00:55:11,942 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__7rgsq9a0vr2e6ivp4n42hiip69c7re11emhk694hgf8mbvjmdj6lha4iqqm4c54s/family1/be354e1570584b5386ed15a9afc375a1 into cfb81dcb75adebd110154b74cfdaab0a/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/d0caf53bc40b4e45a0bff39fc9e0f77b_SeqId_53_) 2023-02-13 00:55:11,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__7rgsq9a0vr2e6ivp4n42hiip69c7re11emhk694hgf8mbvjmdj6lha4iqqm4c54s/family1/be354e1570584b5386ed15a9afc375a1 2023-02-13 00:55:11,943 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 00:55:11,944 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:11,945 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511200f319043c365459dbaeb7588ae4fa35c to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511200f319043c365459dbaeb7588ae4fa35c 2023-02-13 00:55:11,946 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128e05ec97c1a05428f8463cf3c4b0e7930 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128e05ec97c1a05428f8463cf3c4b0e7930 2023-02-13 00:55:11,946 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-02-13 00:55:11,946 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-13 00:55:11,947 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:11,947 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:11,947 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:11,947 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 00:55:11,947 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:55:11,948 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 00:55:11,949 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 881 msec 2023-02-13 00:55:11,995 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-02-13 00:55:11,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure2.ProcedureExecutor(1029): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:12,404 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-02-13 00:55:12,405 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-13 00:55:12,405 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-02-13 00:55:12,407 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-02-13 00:55:12,409 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=cfb81dcb75adebd110154b74cfdaab0a, REOPEN/MOVE}] 2023-02-13 00:55:12,409 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=cfb81dcb75adebd110154b74cfdaab0a, REOPEN/MOVE 2023-02-13 00:55:12,410 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=cfb81dcb75adebd110154b74cfdaab0a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:12,410 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676249712410"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249712410"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249712410"}]},"ts":"1676249712410"} 2023-02-13 00:55:12,411 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure cfb81dcb75adebd110154b74cfdaab0a, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:55:12,563 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:55:12,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing cfb81dcb75adebd110154b74cfdaab0a, disabling compactions & flushes 2023-02-13 00:55:12,563 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:12,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:12,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. after waiting 0 ms 2023-02-13 00:55:12,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:12,568 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-13 00:55:12,569 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:12,569 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for cfb81dcb75adebd110154b74cfdaab0a: Waiting for close lock at 1676249712563Running coprocessor pre-close hooks at 1676249712563Disabling compacts and flushes for region at 1676249712563Disabling writes for close at 1676249712563Writing region close event to WAL at 1676249712565 (+2 ms)Running coprocessor post-close hooks at 1676249712569 (+4 ms)Closed at 1676249712569 2023-02-13 00:55:12,569 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: cfb81dcb75adebd110154b74cfdaab0a to self. 2023-02-13 00:55:12,570 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:55:12,570 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=cfb81dcb75adebd110154b74cfdaab0a, regionState=CLOSED 2023-02-13 00:55:12,570 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676249712570"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249712570"}]},"ts":"1676249712570"} 2023-02-13 00:55:12,572 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=68, resume processing ppid=67 2023-02-13 00:55:12,572 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure cfb81dcb75adebd110154b74cfdaab0a, server=jenkins-hbase4.apache.org,36605,1676249622318 in 160 msec 2023-02-13 00:55:12,572 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=cfb81dcb75adebd110154b74cfdaab0a, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=true 2023-02-13 00:55:12,723 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=cfb81dcb75adebd110154b74cfdaab0a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:12,723 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676249712723"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249712723"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249712723"}]},"ts":"1676249712723"} 2023-02-13 00:55:12,725 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure cfb81dcb75adebd110154b74cfdaab0a, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:55:12,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:12,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => cfb81dcb75adebd110154b74cfdaab0a, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:55:12,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:55:12,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:12,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:55:12,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:55:12,880 INFO [StoreOpener-cfb81dcb75adebd110154b74cfdaab0a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:55:12,881 DEBUG [StoreOpener-cfb81dcb75adebd110154b74cfdaab0a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1 2023-02-13 00:55:12,881 DEBUG [StoreOpener-cfb81dcb75adebd110154b74cfdaab0a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1 2023-02-13 00:55:12,881 INFO [StoreOpener-cfb81dcb75adebd110154b74cfdaab0a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cfb81dcb75adebd110154b74cfdaab0a columnFamilyName family1 2023-02-13 00:55:12,886 DEBUG [StoreOpener-cfb81dcb75adebd110154b74cfdaab0a-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/ab0387741cb24f188ca49511d9a919c6 2023-02-13 00:55:12,889 DEBUG [StoreOpener-cfb81dcb75adebd110154b74cfdaab0a-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/family1/d0caf53bc40b4e45a0bff39fc9e0f77b_SeqId_53_ 2023-02-13 00:55:12,889 INFO [StoreOpener-cfb81dcb75adebd110154b74cfdaab0a-1] regionserver.HStore(310): Store=cfb81dcb75adebd110154b74cfdaab0a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:12,889 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:55:12,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:55:12,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:55:12,893 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened cfb81dcb75adebd110154b74cfdaab0a; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64769032, jitterRate=-0.03486621379852295}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 00:55:12,893 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for cfb81dcb75adebd110154b74cfdaab0a: Running coprocessor pre-open hook at 1676249712879Writing region info on filesystem at 1676249712879Initializing all the Stores at 1676249712880 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} at 1676249712880Cleaning up temporary data from old regions at 1676249712891 (+11 ms)Cleaning up detritus from prior splits at 1676249712891Running coprocessor post-open hooks at 1676249712893 (+2 ms)Region opened successfully at 1676249712893 2023-02-13 00:55:12,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a., pid=69, masterSystemTime=1676249712876 2023-02-13 00:55:12,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:12,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:55:12,895 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=cfb81dcb75adebd110154b74cfdaab0a, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:12,895 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676249712895"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249712895"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249712895"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249712895"}]},"ts":"1676249712895"} 2023-02-13 00:55:12,897 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=67 2023-02-13 00:55:12,897 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure cfb81dcb75adebd110154b74cfdaab0a, server=jenkins-hbase4.apache.org,36605,1676249622318 in 171 msec 2023-02-13 00:55:12,899 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=66 2023-02-13 00:55:12,899 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=cfb81dcb75adebd110154b74cfdaab0a, REOPEN/MOVE in 488 msec 2023-02-13 00:55:12,900 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-13 00:55:12,900 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 492 msec 2023-02-13 00:55:12,901 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 906 msec 2023-02-13 00:55:12,902 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=65 2023-02-13 00:55:12,903 INFO [Listener at localhost/37869] compactions.TestMobCompactor(278): alter status finished 2023-02-13 00:55:12,903 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43563] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 00:55:12,904 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] procedure2.ProcedureExecutor(1029): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 00:55:12,905 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 00:55:12,906 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:55:12,906 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676250312906 2023-02-13 00:55:12,906 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 00:55:12,907 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-13 00:55:12,907 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 00:55:12,907 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] zookeeper.ReadOnlyZKClient(139): Connect 0x739763fc to 127.0.0.1:54478 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:55:12,912 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@70d3d62, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:55:12,912 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:12,912 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:12,912 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:12,912 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:12,912 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:12,912 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:12,912 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:12,912 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 00:55:12,912 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 00:55:12,912 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:55:12,913 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 00:55:12,914 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 9 msec 2023-02-13 00:55:12,972 INFO [Listener at localhost/37869] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=251 (was 245) - Thread LEAK? -, OpenFileDescriptor=643 (was 620) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=211 (was 233), ProcessCount=167 (was 167), AvailableMemoryMB=3479 (was 3501) 2023-02-13 00:55:12,982 INFO [Listener at localhost/37869] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=251, OpenFileDescriptor=643, MaxFileDescriptor=60000, SystemLoadAverage=211, ProcessCount=167, AvailableMemoryMB=3478 2023-02-13 00:55:12,985 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-13 00:55:12,986 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-02-13 00:55:12,987 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 00:55:12,987 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-02-13 00:55:12,988 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 00:55:12,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-13 00:55:12,989 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3 2023-02-13 00:55:12,989 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3 empty. 2023-02-13 00:55:12,990 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3 2023-02-13 00:55:12,990 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-02-13 00:55:13,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-13 00:55:13,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-13 00:55:13,397 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-02-13 00:55:13,398 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7671): creating {ENCODED => afe7f0913f3e084bdd221384f6e196e3, NAME => 'testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp 2023-02-13 00:55:13,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-13 00:55:13,803 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(865): Instantiated testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:13,803 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1603): Closing afe7f0913f3e084bdd221384f6e196e3, disabling compactions & flushes 2023-02-13 00:55:13,803 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1625): Closing region testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3. 2023-02-13 00:55:13,803 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3. 2023-02-13 00:55:13,804 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1713): Acquired close lock on testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3. after waiting 1 ms 2023-02-13 00:55:13,804 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1723): Updates disabled for region testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3. 2023-02-13 00:55:13,804 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1837): Closed testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3. 2023-02-13 00:55:13,804 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1557): Region close journal for afe7f0913f3e084bdd221384f6e196e3: Waiting for close lock at 1676249713803Disabling compacts and flushes for region at 1676249713803Disabling writes for close at 1676249713804 (+1 ms)Writing region close event to WAL at 1676249713804Closed at 1676249713804 2023-02-13 00:55:13,806 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 00:55:13,806 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676249713806"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249713806"}]},"ts":"1676249713806"} 2023-02-13 00:55:13,808 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 00:55:13,808 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 00:55:13,808 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249713808"}]},"ts":"1676249713808"} 2023-02-13 00:55:13,809 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-02-13 00:55:13,812 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=afe7f0913f3e084bdd221384f6e196e3, ASSIGN}] 2023-02-13 00:55:13,814 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=afe7f0913f3e084bdd221384f6e196e3, ASSIGN 2023-02-13 00:55:13,814 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=afe7f0913f3e084bdd221384f6e196e3, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=false 2023-02-13 00:55:13,965 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=afe7f0913f3e084bdd221384f6e196e3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:13,965 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676249713965"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249713965"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249713965"}]},"ts":"1676249713965"} 2023-02-13 00:55:13,967 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure afe7f0913f3e084bdd221384f6e196e3, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:55:14,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-13 00:55:14,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3. 2023-02-13 00:55:14,121 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => afe7f0913f3e084bdd221384f6e196e3, NAME => 'testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:55:14,121 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-02-13 00:55:14,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-02-13 00:55:14,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction afe7f0913f3e084bdd221384f6e196e3 2023-02-13 00:55:14,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:14,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for afe7f0913f3e084bdd221384f6e196e3 2023-02-13 00:55:14,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for afe7f0913f3e084bdd221384f6e196e3 2023-02-13 00:55:14,124 INFO [StoreOpener-afe7f0913f3e084bdd221384f6e196e3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region afe7f0913f3e084bdd221384f6e196e3 2023-02-13 00:55:14,125 DEBUG [StoreOpener-afe7f0913f3e084bdd221384f6e196e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1 2023-02-13 00:55:14,125 DEBUG [StoreOpener-afe7f0913f3e084bdd221384f6e196e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1 2023-02-13 00:55:14,125 INFO [StoreOpener-afe7f0913f3e084bdd221384f6e196e3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region afe7f0913f3e084bdd221384f6e196e3 columnFamilyName f1 2023-02-13 00:55:14,126 INFO [StoreOpener-afe7f0913f3e084bdd221384f6e196e3-1] regionserver.HStore(310): Store=afe7f0913f3e084bdd221384f6e196e3/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:14,127 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3 2023-02-13 00:55:14,127 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3 2023-02-13 00:55:14,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for afe7f0913f3e084bdd221384f6e196e3 2023-02-13 00:55:14,131 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:55:14,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened afe7f0913f3e084bdd221384f6e196e3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64725595, jitterRate=-0.03551347553730011}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 00:55:14,131 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for afe7f0913f3e084bdd221384f6e196e3: Running coprocessor pre-open hook at 1676249714123Writing region info on filesystem at 1676249714123Initializing all the Stores at 1676249714123Instantiating store for column family {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249714123Cleaning up temporary data from old regions at 1676249714128 (+5 ms)Cleaning up detritus from prior splits at 1676249714128Running coprocessor post-open hooks at 1676249714131 (+3 ms)Region opened successfully at 1676249714131 2023-02-13 00:55:14,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3., pid=73, masterSystemTime=1676249714118 2023-02-13 00:55:14,133 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3. 2023-02-13 00:55:14,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3. 2023-02-13 00:55:14,134 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=afe7f0913f3e084bdd221384f6e196e3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:14,134 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676249714134"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249714134"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249714134"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249714134"}]},"ts":"1676249714134"} 2023-02-13 00:55:14,136 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-13 00:55:14,136 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure afe7f0913f3e084bdd221384f6e196e3, server=jenkins-hbase4.apache.org,36605,1676249622318 in 168 msec 2023-02-13 00:55:14,137 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=72, resume processing ppid=71 2023-02-13 00:55:14,137 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=afe7f0913f3e084bdd221384f6e196e3, ASSIGN in 324 msec 2023-02-13 00:55:14,138 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 00:55:14,138 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249714138"}]},"ts":"1676249714138"} 2023-02-13 00:55:14,139 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-02-13 00:55:14,142 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 00:55:14,143 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 1.1570 sec 2023-02-13 00:55:15,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-13 00:55:15,092 INFO [Listener at localhost/37869] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-02-13 00:55:15,092 DEBUG [Listener at localhost/37869] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-02-13 00:55:15,094 INFO [Listener at localhost/37869] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 00:55:15,097 INFO [Listener at localhost/37869] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-02-13 00:55:15,097 INFO [Listener at localhost/37869] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 00:55:15,097 INFO [Listener at localhost/37869] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-02-13 00:55:15,101 INFO [Listener at localhost/37869] regionserver.HRegion(2744): Flushing afe7f0913f3e084bdd221384f6e196e3 1/1 column families, dataSize=37 B heapSize=368 B 2023-02-13 00:55:15,512 INFO [Listener at localhost/37869] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213764df84e77044132907e55e0a28648b6 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230213764df84e77044132907e55e0a28648b6 2023-02-13 00:55:15,517 INFO [Listener at localhost/37869] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/.tmp/f1/6034740cfba7460482a57b87cced6b90 2023-02-13 00:55:15,522 DEBUG [Listener at localhost/37869] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/.tmp/f1/6034740cfba7460482a57b87cced6b90 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90 2023-02-13 00:55:15,526 INFO [Listener at localhost/37869] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90, entries=1, sequenceid=5, filesize=4.9 K 2023-02-13 00:55:15,527 INFO [Listener at localhost/37869] regionserver.HRegion(2947): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for afe7f0913f3e084bdd221384f6e196e3 in 426ms, sequenceid=5, compaction requested=false 2023-02-13 00:55:15,527 DEBUG [Listener at localhost/37869] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-02-13 00:55:15,527 DEBUG [Listener at localhost/37869] regionserver.HRegion(2445): Flush status journal for afe7f0913f3e084bdd221384f6e196e3: 2023-02-13 00:55:15,529 INFO [Listener at localhost/37869] regionserver.HRegion(2744): Flushing afe7f0913f3e084bdd221384f6e196e3 1/1 column families, dataSize=37 B heapSize=368 B 2023-02-13 00:55:15,940 INFO [Listener at localhost/37869] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021364b40706e47d46249feed210c2b8d5f4 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023021364b40706e47d46249feed210c2b8d5f4 2023-02-13 00:55:16,268 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 00:55:16,346 INFO [Listener at localhost/37869] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/.tmp/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2 2023-02-13 00:55:16,350 DEBUG [Listener at localhost/37869] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/.tmp/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2 2023-02-13 00:55:16,353 INFO [Listener at localhost/37869] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2, entries=1, sequenceid=9, filesize=4.9 K 2023-02-13 00:55:16,354 INFO [Listener at localhost/37869] regionserver.HRegion(2947): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for afe7f0913f3e084bdd221384f6e196e3 in 825ms, sequenceid=9, compaction requested=false 2023-02-13 00:55:16,354 DEBUG [Listener at localhost/37869] regionserver.HRegion(2445): Flush status journal for afe7f0913f3e084bdd221384f6e196e3: 2023-02-13 00:55:16,355 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43563] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-02-13 00:55:16,357 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] procedure2.ProcedureExecutor(1029): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-13 00:55:16,357 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-13 00:55:16,358 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:55:16,358 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676250316358 2023-02-13 00:55:16,358 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-13 00:55:16,358 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-02-13 00:55:16,358 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-02-13 00:55:16,359 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] zookeeper.ReadOnlyZKClient(139): Connect 0x0e91d8d3 to 127.0.0.1:54478 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:55:16,363 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6019124e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:55:16,363 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230213 for table='testGetAfterCompaction' and column='f1' 2023-02-13 00:55:16,407 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:16,407 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:16,457 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:16,458 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:16,508 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:16,508 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:16,559 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:16,559 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:16,610 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:16,610 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:16,661 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:16,661 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:16,712 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:16,712 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:16,762 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:16,762 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:16,813 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:16,813 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:16,864 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:16,864 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:16,914 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:16,915 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:16,965 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:16,965 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,016 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,016 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,067 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,067 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,118 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,118 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,168 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,169 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,187 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302133592e2dac90c4577b578385e36d418a1 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302133592e2dac90c4577b578385e36d418a1 2023-02-13 00:55:17,188 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='testGetAfterCompaction' column='f1' 2023-02-13 00:55:17,201 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:55:17,202 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37022, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:55:17,207 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 00:55:17,209 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39610, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 00:55:17,217 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230213/f1/680e1256a1c54950af41432c113d642e first=Optional[r1] last=Optional[r2] 2023-02-13 00:55:17,217 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row with hfile group [{f1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230213/f1/680e1256a1c54950af41432c113d642e}] 2023-02-13 00:55:17,220 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,220 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,221 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230213/f1/680e1256a1c54950af41432c113d642e for inclusion in afe7f0913f3e084bdd221384f6e196e3/f1 2023-02-13 00:55:17,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-02-13 00:55:17,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 00:55:17,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for afe7f0913f3e084bdd221384f6e196e3: 2023-02-13 00:55:17,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230213/f1/680e1256a1c54950af41432c113d642e to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testGetAfterCompaction__dpmcjd5r689vrnttq3fqmo8ulsf9er7ja0ala7h6e5ok2gav1gdqhek0r6mgtgb1/f1/680e1256a1c54950af41432c113d642e 2023-02-13 00:55:17,225 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testGetAfterCompaction__dpmcjd5r689vrnttq3fqmo8ulsf9er7ja0ala7h6e5ok2gav1gdqhek0r6mgtgb1/f1/680e1256a1c54950af41432c113d642e as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_ 2023-02-13 00:55:17,226 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testGetAfterCompaction__dpmcjd5r689vrnttq3fqmo8ulsf9er7ja0ala7h6e5ok2gav1gdqhek0r6mgtgb1/f1/680e1256a1c54950af41432c113d642e into afe7f0913f3e084bdd221384f6e196e3/f1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_ - updating store file list. 2023-02-13 00:55:17,229 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_ into afe7f0913f3e084bdd221384f6e196e3/f1 2023-02-13 00:55:17,229 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testGetAfterCompaction__dpmcjd5r689vrnttq3fqmo8ulsf9er7ja0ala7h6e5ok2gav1gdqhek0r6mgtgb1/f1/680e1256a1c54950af41432c113d642e into afe7f0913f3e084bdd221384f6e196e3/f1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_) 2023-02-13 00:55:17,229 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testGetAfterCompaction__dpmcjd5r689vrnttq3fqmo8ulsf9er7ja0ala7h6e5ok2gav1gdqhek0r6mgtgb1/f1/680e1256a1c54950af41432c113d642e 2023-02-13 00:55:17,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(1912): afe7f0913f3e084bdd221384f6e196e3/f1 is initiating minor compaction 2023-02-13 00:55:17,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@413e4ef5; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:55:17,230 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of afe7f0913f3e084bdd221384f6e196e3/f1 in testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3. 2023-02-13 00:55:17,230 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/.tmp, totalSize=10.0 K 2023-02-13 00:55:17,230 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 9c7cb87c2911428d8a2aedc7a2f6b9c2, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-13 00:55:17,230 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting c20024700c074304af1d5082fb32b14e_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-13 00:55:17,232 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='testGetAfterCompaction' column='f1' 2023-02-13 00:55:17,233 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:17,234 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023021364b40706e47d46249feed210c2b8d5f4 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023021364b40706e47d46249feed210c2b8d5f4 2023-02-13 00:55:17,235 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230213764df84e77044132907e55e0a28648b6 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230213764df84e77044132907e55e0a28648b6 2023-02-13 00:55:17,235 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-02-13 00:55:17,235 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230213' 2023-02-13 00:55:17,236 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230213 for table='testGetAfterCompaction' and column='f1' 2023-02-13 00:55:17,236 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-02-13 00:55:17,236 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:55:17,237 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-13 00:55:17,238 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 881 msec 2023-02-13 00:55:17,238 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] throttle.PressureAwareThroughputController(145): afe7f0913f3e084bdd221384f6e196e3#f1#compaction#76 average throughput is 0.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:55:17,270 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,271 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,271 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676249717183; access_time=1676249716373; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 00:55:17,274 DEBUG [Listener at localhost/37869] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 00:55:17,277 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37028, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 00:55:17,277 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36605] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3. 2023-02-13 00:55:17,278 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36605] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-02-13 00:55:17,278 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36605] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-13 00:55:17,278 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36605] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-13 00:55:17,278 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36605] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3. because compaction request was cancelled 2023-02-13 00:55:17,330 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,330 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,330 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676249717183; access_time=1676249716373; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 00:55:17,381 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,381 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,381 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676249717183; access_time=1676249716373; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 00:55:17,431 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,432 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,432 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676249717183; access_time=1676249716373; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 00:55:17,482 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,482 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,482 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676249717183; access_time=1676249716373; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 00:55:17,533 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,533 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,533 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676249717183; access_time=1676249716373; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 00:55:17,584 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,584 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,584 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676249717183; access_time=1676249716373; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 00:55:17,635 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,635 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,635 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676249717183; access_time=1676249716373; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 00:55:17,685 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,685 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,686 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676249717183; access_time=1676249716373; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 00:55:17,736 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,736 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,736 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676249717183; access_time=1676249716373; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 00:55:17,787 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,787 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,787 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676249717183; access_time=1676249716373; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 00:55:17,838 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,838 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,838 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676249717183; access_time=1676249716373; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 00:55:17,889 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,889 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,889 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676249717183; access_time=1676249716373; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 00:55:17,939 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,940 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,940 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676249717183; access_time=1676249716373; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 00:55:17,990 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,990 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:17,990 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676249717183; access_time=1676249716373; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 00:55:18,041 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:18,041 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:18,041 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676249717183; access_time=1676249716373; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 00:55:18,051 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/.tmp/f1/d49c406654b248f4b48b71b3558e2983 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/d49c406654b248f4b48b71b3558e2983 2023-02-13 00:55:18,055 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in afe7f0913f3e084bdd221384f6e196e3/f1 of afe7f0913f3e084bdd221384f6e196e3 into d49c406654b248f4b48b71b3558e2983(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:55:18,055 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for afe7f0913f3e084bdd221384f6e196e3: 2023-02-13 00:55:18,055 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1676249717230; duration=0sec 2023-02-13 00:55:18,055 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:55:18,092 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:18,092 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:18,092 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676249717183; access_time=1676249716373; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 00:55:18,092 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/d49c406654b248f4b48b71b3558e2983; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1676249718046; access_time=1676249717235; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:18,143 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:18,143 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249716345; access_time=1676249715531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:18,143 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676249717183; access_time=1676249716373; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 00:55:18,143 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/d49c406654b248f4b48b71b3558e2983; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1676249718046; access_time=1676249717235; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:18,150 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_] to archive 2023-02-13 00:55:18,151 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:18,152 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/9c7cb87c2911428d8a2aedc7a2f6b9c2 2023-02-13 00:55:18,153 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_ to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/c20024700c074304af1d5082fb32b14e_SeqId_12_ 2023-02-13 00:55:18,193 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676249715517; access_time=1676249715104; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:18,194 INFO [Listener at localhost/37869] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/d49c406654b248f4b48b71b3558e2983; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1676249718046; access_time=1676249717235; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 00:55:18,313 INFO [Listener at localhost/37869] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-02-13 00:55:18,317 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testGetAfterCompaction 2023-02-13 00:55:18,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure2.ProcedureExecutor(1029): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-02-13 00:55:18,325 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249718325"}]},"ts":"1676249718325"} 2023-02-13 00:55:18,326 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-02-13 00:55:18,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-13 00:55:18,328 INFO [PEWorker-5] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-02-13 00:55:18,329 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=afe7f0913f3e084bdd221384f6e196e3, UNASSIGN}] 2023-02-13 00:55:18,330 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=afe7f0913f3e084bdd221384f6e196e3, UNASSIGN 2023-02-13 00:55:18,331 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=afe7f0913f3e084bdd221384f6e196e3, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:18,331 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676249718331"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249718331"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249718331"}]},"ts":"1676249718331"} 2023-02-13 00:55:18,332 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure afe7f0913f3e084bdd221384f6e196e3, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:55:18,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-13 00:55:18,483 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close afe7f0913f3e084bdd221384f6e196e3 2023-02-13 00:55:18,483 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing afe7f0913f3e084bdd221384f6e196e3, disabling compactions & flushes 2023-02-13 00:55:18,483 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3. 2023-02-13 00:55:18,483 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3. 2023-02-13 00:55:18,483 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3. after waiting 0 ms 2023-02-13 00:55:18,483 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3. 2023-02-13 00:55:18,487 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-02-13 00:55:18,487 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-02-13 00:55:18,488 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3. 2023-02-13 00:55:18,489 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for afe7f0913f3e084bdd221384f6e196e3: Waiting for close lock at 1676249718483Running coprocessor pre-close hooks at 1676249718483Disabling compacts and flushes for region at 1676249718483Disabling writes for close at 1676249718483Writing region close event to WAL at 1676249718484 (+1 ms)Running coprocessor post-close hooks at 1676249718487 (+3 ms)Closed at 1676249718488 (+1 ms) 2023-02-13 00:55:18,490 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed afe7f0913f3e084bdd221384f6e196e3 2023-02-13 00:55:18,490 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=afe7f0913f3e084bdd221384f6e196e3, regionState=CLOSED 2023-02-13 00:55:18,490 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676249718490"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249718490"}]},"ts":"1676249718490"} 2023-02-13 00:55:18,493 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-02-13 00:55:18,493 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure afe7f0913f3e084bdd221384f6e196e3, server=jenkins-hbase4.apache.org,36605,1676249622318 in 159 msec 2023-02-13 00:55:18,494 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=75 2023-02-13 00:55:18,494 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=afe7f0913f3e084bdd221384f6e196e3, UNASSIGN in 164 msec 2023-02-13 00:55:18,495 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249718494"}]},"ts":"1676249718494"} 2023-02-13 00:55:18,495 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-02-13 00:55:18,497 INFO [PEWorker-5] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-02-13 00:55:18,498 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 178 msec 2023-02-13 00:55:18,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-13 00:55:18,628 INFO [Listener at localhost/37869] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-02-13 00:55:18,634 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testGetAfterCompaction 2023-02-13 00:55:18,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure2.ProcedureExecutor(1029): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-02-13 00:55:18,641 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-13 00:55:18,642 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-13 00:55:18,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-02-13 00:55:18,645 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3 2023-02-13 00:55:18,647 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1, FileablePath, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/recovered.edits] 2023-02-13 00:55:18,649 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/6034740cfba7460482a57b87cced6b90 2023-02-13 00:55:18,650 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/d49c406654b248f4b48b71b3558e2983 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/f1/d49c406654b248f4b48b71b3558e2983 2023-02-13 00:55:18,652 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/recovered.edits/15.seqid to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3/recovered.edits/15.seqid 2023-02-13 00:55:18,653 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testGetAfterCompaction/afe7f0913f3e084bdd221384f6e196e3 2023-02-13 00:55:18,653 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-02-13 00:55:18,653 DEBUG [PEWorker-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-02-13 00:55:18,654 DEBUG [PEWorker-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-02-13 00:55:18,656 DEBUG [PEWorker-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302133592e2dac90c4577b578385e36d418a1 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302133592e2dac90c4577b578385e36d418a1 2023-02-13 00:55:18,656 DEBUG [PEWorker-3] backup.HFileArchiver(596): Deleted hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-02-13 00:55:18,659 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-13 00:55:18,679 WARN [PEWorker-3] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-02-13 00:55:18,681 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-02-13 00:55:18,683 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-13 00:55:18,683 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-02-13 00:55:18,683 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249718683"}]},"ts":"9223372036854775807"} 2023-02-13 00:55:18,684 INFO [PEWorker-3] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-13 00:55:18,684 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => afe7f0913f3e084bdd221384f6e196e3, NAME => 'testGetAfterCompaction,,1676249712985.afe7f0913f3e084bdd221384f6e196e3.', STARTKEY => '', ENDKEY => ''}] 2023-02-13 00:55:18,684 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-02-13 00:55:18,684 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676249718684"}]},"ts":"9223372036854775807"} 2023-02-13 00:55:18,686 INFO [PEWorker-3] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-02-13 00:55:18,692 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-13 00:55:18,693 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 57 msec 2023-02-13 00:55:18,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-02-13 00:55:18,744 INFO [Listener at localhost/37869] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-02-13 00:55:18,757 INFO [Listener at localhost/37869] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=257 (was 251) - Thread LEAK? -, OpenFileDescriptor=649 (was 643) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=194 (was 211), ProcessCount=167 (was 167), AvailableMemoryMB=3474 (was 3478) 2023-02-13 00:55:18,767 INFO [Listener at localhost/37869] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=257, OpenFileDescriptor=649, MaxFileDescriptor=60000, SystemLoadAverage=194, ProcessCount=167, AvailableMemoryMB=3474 2023-02-13 00:55:18,769 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-13 00:55:18,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-02-13 00:55:18,771 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 00:55:18,771 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-02-13 00:55:18,771 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 00:55:18,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 00:55:18,773 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013 2023-02-13 00:55:18,774 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d 2023-02-13 00:55:18,774 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3 2023-02-13 00:55:18,774 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013 empty. 2023-02-13 00:55:18,774 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3 empty. 2023-02-13 00:55:18,774 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013 2023-02-13 00:55:18,774 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d empty. 2023-02-13 00:55:18,775 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3 2023-02-13 00:55:18,775 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d 2023-02-13 00:55:18,775 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-02-13 00:55:18,873 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 00:55:19,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 00:55:19,183 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-02-13 00:55:19,184 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7671): creating {ENCODED => 63f9bed78ea4a19108ec7ecea54182e3, NAME => 'testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp 2023-02-13 00:55:19,185 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7671): creating {ENCODED => 89b71b72a624c785439c7640cd6fb013, NAME => 'testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp 2023-02-13 00:55:19,185 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7671): creating {ENCODED => 6b485aea7037b0c8ec399d8612e4ff1d, NAME => 'testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp 2023-02-13 00:55:19,201 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:19,201 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1603): Closing 6b485aea7037b0c8ec399d8612e4ff1d, disabling compactions & flushes 2023-02-13 00:55:19,201 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:19,201 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:19,201 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. after waiting 0 ms 2023-02-13 00:55:19,201 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:19,201 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:19,201 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1557): Region close journal for 6b485aea7037b0c8ec399d8612e4ff1d: Waiting for close lock at 1676249719201Disabling compacts and flushes for region at 1676249719201Disabling writes for close at 1676249719201Writing region close event to WAL at 1676249719201Closed at 1676249719201 2023-02-13 00:55:19,374 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 00:55:19,599 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:19,599 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1603): Closing 63f9bed78ea4a19108ec7ecea54182e3, disabling compactions & flushes 2023-02-13 00:55:19,599 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:19,599 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:19,599 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. after waiting 0 ms 2023-02-13 00:55:19,599 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:19,599 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:19,599 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1557): Region close journal for 63f9bed78ea4a19108ec7ecea54182e3: Waiting for close lock at 1676249719599Disabling compacts and flushes for region at 1676249719599Disabling writes for close at 1676249719599Writing region close event to WAL at 1676249719599Closed at 1676249719599 2023-02-13 00:55:19,603 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:19,603 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1603): Closing 89b71b72a624c785439c7640cd6fb013, disabling compactions & flushes 2023-02-13 00:55:19,603 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:19,603 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:19,603 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. after waiting 0 ms 2023-02-13 00:55:19,603 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:19,603 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:19,603 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1557): Region close journal for 89b71b72a624c785439c7640cd6fb013: Waiting for close lock at 1676249719603Disabling compacts and flushes for region at 1676249719603Disabling writes for close at 1676249719603Writing region close event to WAL at 1676249719603Closed at 1676249719603 2023-02-13 00:55:19,605 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 00:55:19,606 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676249719606"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249719606"}]},"ts":"1676249719606"} 2023-02-13 00:55:19,606 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676249719606"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249719606"}]},"ts":"1676249719606"} 2023-02-13 00:55:19,606 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676249719606"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249719606"}]},"ts":"1676249719606"} 2023-02-13 00:55:19,608 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-13 00:55:19,609 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 00:55:19,609 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249719609"}]},"ts":"1676249719609"} 2023-02-13 00:55:19,610 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-02-13 00:55:19,614 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=89b71b72a624c785439c7640cd6fb013, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6b485aea7037b0c8ec399d8612e4ff1d, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=63f9bed78ea4a19108ec7ecea54182e3, ASSIGN}] 2023-02-13 00:55:19,616 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=89b71b72a624c785439c7640cd6fb013, ASSIGN 2023-02-13 00:55:19,616 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=63f9bed78ea4a19108ec7ecea54182e3, ASSIGN 2023-02-13 00:55:19,616 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6b485aea7037b0c8ec399d8612e4ff1d, ASSIGN 2023-02-13 00:55:19,616 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6b485aea7037b0c8ec399d8612e4ff1d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=false 2023-02-13 00:55:19,616 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=89b71b72a624c785439c7640cd6fb013, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=false 2023-02-13 00:55:19,616 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=63f9bed78ea4a19108ec7ecea54182e3, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=false 2023-02-13 00:55:19,768 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=89b71b72a624c785439c7640cd6fb013, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:19,768 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=63f9bed78ea4a19108ec7ecea54182e3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:19,768 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=6b485aea7037b0c8ec399d8612e4ff1d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:19,768 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676249719768"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249719768"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249719768"}]},"ts":"1676249719768"} 2023-02-13 00:55:19,768 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676249719768"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249719768"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249719768"}]},"ts":"1676249719768"} 2023-02-13 00:55:19,768 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676249719768"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249719768"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249719768"}]},"ts":"1676249719768"} 2023-02-13 00:55:19,770 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; OpenRegionProcedure 89b71b72a624c785439c7640cd6fb013, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:55:19,770 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=84, ppid=81, state=RUNNABLE; OpenRegionProcedure 6b485aea7037b0c8ec399d8612e4ff1d, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:55:19,772 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=85, ppid=82, state=RUNNABLE; OpenRegionProcedure 63f9bed78ea4a19108ec7ecea54182e3, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:55:19,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 00:55:19,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:19,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 63f9bed78ea4a19108ec7ecea54182e3, NAME => 'testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3.', STARTKEY => '2', ENDKEY => ''} 2023-02-13 00:55:19,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 63f9bed78ea4a19108ec7ecea54182e3 2023-02-13 00:55:19,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:19,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 63f9bed78ea4a19108ec7ecea54182e3 2023-02-13 00:55:19,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 63f9bed78ea4a19108ec7ecea54182e3 2023-02-13 00:55:19,927 INFO [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 63f9bed78ea4a19108ec7ecea54182e3 2023-02-13 00:55:19,928 DEBUG [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1 2023-02-13 00:55:19,928 DEBUG [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1 2023-02-13 00:55:19,928 INFO [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 63f9bed78ea4a19108ec7ecea54182e3 columnFamilyName family1 2023-02-13 00:55:19,929 INFO [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] regionserver.HStore(310): Store=63f9bed78ea4a19108ec7ecea54182e3/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:19,930 INFO [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 63f9bed78ea4a19108ec7ecea54182e3 2023-02-13 00:55:19,931 DEBUG [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2 2023-02-13 00:55:19,931 DEBUG [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2 2023-02-13 00:55:19,931 INFO [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 63f9bed78ea4a19108ec7ecea54182e3 columnFamilyName family2 2023-02-13 00:55:19,932 INFO [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] regionserver.HStore(310): Store=63f9bed78ea4a19108ec7ecea54182e3/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:19,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3 2023-02-13 00:55:19,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3 2023-02-13 00:55:19,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:55:19,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 63f9bed78ea4a19108ec7ecea54182e3 2023-02-13 00:55:19,937 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:55:19,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 63f9bed78ea4a19108ec7ecea54182e3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69297793, jitterRate=0.032617583870887756}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:55:19,938 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 63f9bed78ea4a19108ec7ecea54182e3: Running coprocessor pre-open hook at 1676249719926Writing region info on filesystem at 1676249719926Initializing all the Stores at 1676249719926Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249719926Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249719927 (+1 ms)Cleaning up temporary data from old regions at 1676249719934 (+7 ms)Cleaning up detritus from prior splits at 1676249719934Running coprocessor post-open hooks at 1676249719938 (+4 ms)Region opened successfully at 1676249719938 2023-02-13 00:55:19,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3., pid=85, masterSystemTime=1676249719922 2023-02-13 00:55:19,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:19,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:19,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:19,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6b485aea7037b0c8ec399d8612e4ff1d, NAME => 'testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d.', STARTKEY => '1', ENDKEY => '2'} 2023-02-13 00:55:19,940 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=63f9bed78ea4a19108ec7ecea54182e3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:19,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 6b485aea7037b0c8ec399d8612e4ff1d 2023-02-13 00:55:19,940 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676249719940"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249719940"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249719940"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249719940"}]},"ts":"1676249719940"} 2023-02-13 00:55:19,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:19,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 6b485aea7037b0c8ec399d8612e4ff1d 2023-02-13 00:55:19,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 6b485aea7037b0c8ec399d8612e4ff1d 2023-02-13 00:55:19,941 INFO [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6b485aea7037b0c8ec399d8612e4ff1d 2023-02-13 00:55:19,943 DEBUG [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1 2023-02-13 00:55:19,943 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=85, resume processing ppid=82 2023-02-13 00:55:19,943 DEBUG [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1 2023-02-13 00:55:19,943 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=85, ppid=82, state=SUCCESS; OpenRegionProcedure 63f9bed78ea4a19108ec7ecea54182e3, server=jenkins-hbase4.apache.org,36605,1676249622318 in 170 msec 2023-02-13 00:55:19,943 INFO [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6b485aea7037b0c8ec399d8612e4ff1d columnFamilyName family1 2023-02-13 00:55:19,944 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=63f9bed78ea4a19108ec7ecea54182e3, ASSIGN in 329 msec 2023-02-13 00:55:19,944 INFO [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] regionserver.HStore(310): Store=6b485aea7037b0c8ec399d8612e4ff1d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:19,944 INFO [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 6b485aea7037b0c8ec399d8612e4ff1d 2023-02-13 00:55:19,945 DEBUG [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2 2023-02-13 00:55:19,945 DEBUG [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2 2023-02-13 00:55:19,945 INFO [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6b485aea7037b0c8ec399d8612e4ff1d columnFamilyName family2 2023-02-13 00:55:19,946 INFO [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] regionserver.HStore(310): Store=6b485aea7037b0c8ec399d8612e4ff1d/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:19,947 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d 2023-02-13 00:55:19,947 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d 2023-02-13 00:55:19,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:55:19,949 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 6b485aea7037b0c8ec399d8612e4ff1d 2023-02-13 00:55:19,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:55:19,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 6b485aea7037b0c8ec399d8612e4ff1d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75025242, jitterRate=0.11796322464942932}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:55:19,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 6b485aea7037b0c8ec399d8612e4ff1d: Running coprocessor pre-open hook at 1676249719941Writing region info on filesystem at 1676249719941Initializing all the Stores at 1676249719941Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249719941Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249719941Cleaning up temporary data from old regions at 1676249719948 (+7 ms)Cleaning up detritus from prior splits at 1676249719948Running coprocessor post-open hooks at 1676249719951 (+3 ms)Region opened successfully at 1676249719951 2023-02-13 00:55:19,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d., pid=84, masterSystemTime=1676249719922 2023-02-13 00:55:19,953 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:19,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:19,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:19,953 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=6b485aea7037b0c8ec399d8612e4ff1d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:19,953 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 89b71b72a624c785439c7640cd6fb013, NAME => 'testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013.', STARTKEY => '', ENDKEY => '1'} 2023-02-13 00:55:19,953 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676249719953"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249719953"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249719953"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249719953"}]},"ts":"1676249719953"} 2023-02-13 00:55:19,953 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 89b71b72a624c785439c7640cd6fb013 2023-02-13 00:55:19,953 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:19,954 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 89b71b72a624c785439c7640cd6fb013 2023-02-13 00:55:19,954 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 89b71b72a624c785439c7640cd6fb013 2023-02-13 00:55:19,954 INFO [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 89b71b72a624c785439c7640cd6fb013 2023-02-13 00:55:19,956 DEBUG [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1 2023-02-13 00:55:19,956 DEBUG [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1 2023-02-13 00:55:19,956 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=84, resume processing ppid=81 2023-02-13 00:55:19,956 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=84, ppid=81, state=SUCCESS; OpenRegionProcedure 6b485aea7037b0c8ec399d8612e4ff1d, server=jenkins-hbase4.apache.org,36605,1676249622318 in 184 msec 2023-02-13 00:55:19,956 INFO [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 89b71b72a624c785439c7640cd6fb013 columnFamilyName family1 2023-02-13 00:55:19,957 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6b485aea7037b0c8ec399d8612e4ff1d, ASSIGN in 343 msec 2023-02-13 00:55:19,957 INFO [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] regionserver.HStore(310): Store=89b71b72a624c785439c7640cd6fb013/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:19,957 INFO [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 89b71b72a624c785439c7640cd6fb013 2023-02-13 00:55:19,958 DEBUG [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2 2023-02-13 00:55:19,958 DEBUG [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2 2023-02-13 00:55:19,958 INFO [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 89b71b72a624c785439c7640cd6fb013 columnFamilyName family2 2023-02-13 00:55:19,959 INFO [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] regionserver.HStore(310): Store=89b71b72a624c785439c7640cd6fb013/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:19,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013 2023-02-13 00:55:19,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013 2023-02-13 00:55:19,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:55:19,962 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 89b71b72a624c785439c7640cd6fb013 2023-02-13 00:55:19,964 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:55:19,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 89b71b72a624c785439c7640cd6fb013; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74421910, jitterRate=0.10897287726402283}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:55:19,964 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 89b71b72a624c785439c7640cd6fb013: Running coprocessor pre-open hook at 1676249719954Writing region info on filesystem at 1676249719954Initializing all the Stores at 1676249719954Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249719954Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249719954Cleaning up temporary data from old regions at 1676249719960 (+6 ms)Cleaning up detritus from prior splits at 1676249719961 (+1 ms)Running coprocessor post-open hooks at 1676249719964 (+3 ms)Region opened successfully at 1676249719964 2023-02-13 00:55:19,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013., pid=83, masterSystemTime=1676249719922 2023-02-13 00:55:19,966 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:19,966 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:19,967 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=89b71b72a624c785439c7640cd6fb013, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:19,967 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676249719966"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249719966"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249719966"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249719966"}]},"ts":"1676249719966"} 2023-02-13 00:55:19,969 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-13 00:55:19,969 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; OpenRegionProcedure 89b71b72a624c785439c7640cd6fb013, server=jenkins-hbase4.apache.org,36605,1676249622318 in 198 msec 2023-02-13 00:55:19,971 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-02-13 00:55:19,971 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=89b71b72a624c785439c7640cd6fb013, ASSIGN in 356 msec 2023-02-13 00:55:19,971 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 00:55:19,971 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249719971"}]},"ts":"1676249719971"} 2023-02-13 00:55:19,972 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-02-13 00:55:19,974 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 00:55:19,975 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 1.2050 sec 2023-02-13 00:55:20,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 00:55:20,876 INFO [Listener at localhost/37869] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-02-13 00:55:20,883 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:55:20,884 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:55:20,885 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:55:20,890 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:20,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-13 00:55:20,892 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-13 00:55:20,892 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:20,892 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-13 00:55:20,892 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-13 00:55:20,892 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:20,892 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 00:55:20,894 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:20,894 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:20,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:20,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:20,895 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:20,895 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:20,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 00:55:20,895 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:20,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:20,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 00:55:20,895 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-13 00:55:20,896 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-13 00:55:20,896 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-13 00:55:20,896 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:20,896 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-13 00:55:20,898 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:20,898 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. started... 2023-02-13 00:55:20,898 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:20,898 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. started... 2023-02-13 00:55:20,899 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 00:55:20,899 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:20,899 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 6b485aea7037b0c8ec399d8612e4ff1d 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 00:55:20,899 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:20,899 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 63f9bed78ea4a19108ec7ecea54182e3 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 00:55:20,899 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. started... 2023-02-13 00:55:20,900 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 89b71b72a624c785439c7640cd6fb013 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 00:55:21,321 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213348ca6e4353c43259c1482c61f82b4a5 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213348ca6e4353c43259c1482c61f82b4a5 2023-02-13 00:55:21,325 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302136a074e5589c4444cbea4ec51c8c112f7 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302136a074e5589c4444cbea4ec51c8c112f7 2023-02-13 00:55:21,327 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302139ab092340bd14dbc8b723519529420ae to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302139ab092340bd14dbc8b723519529420ae 2023-02-13 00:55:21,727 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp/family1/b764582729ff4ae3a7b761976a0a3c41 2023-02-13 00:55:21,730 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp/family1/f84a259856c34b6695a6bb54160a6af0 2023-02-13 00:55:21,731 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp/family1/5698639ce41540af9d712c378fe62e38 2023-02-13 00:55:22,142 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021301e50cc3adca4f52a6f0a591b7de5f77 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023021301e50cc3adca4f52a6f0a591b7de5f77 2023-02-13 00:55:22,147 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp/family2/a0f7d35e413a49d6bf0acc6f8d6693b6 2023-02-13 00:55:22,148 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021329cbdffeaba74431b13e6040dd8db724 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023021329cbdffeaba74431b13e6040dd8db724 2023-02-13 00:55:22,148 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302134f08f76cbaf1432b9c2e01f3aeffa05c to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302134f08f76cbaf1432b9c2e01f3aeffa05c 2023-02-13 00:55:22,152 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp/family1/b764582729ff4ae3a7b761976a0a3c41 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/b764582729ff4ae3a7b761976a0a3c41 2023-02-13 00:55:22,155 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/b764582729ff4ae3a7b761976a0a3c41, entries=4, sequenceid=5, filesize=5.4 K 2023-02-13 00:55:22,155 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp/family2/a0f7d35e413a49d6bf0acc6f8d6693b6 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/a0f7d35e413a49d6bf0acc6f8d6693b6 2023-02-13 00:55:22,158 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/a0f7d35e413a49d6bf0acc6f8d6693b6, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 00:55:22,159 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 89b71b72a624c785439c7640cd6fb013 in 1259ms, sequenceid=5, compaction requested=false 2023-02-13 00:55:22,159 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-02-13 00:55:22,159 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 89b71b72a624c785439c7640cd6fb013: 2023-02-13 00:55:22,159 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:22,159 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 00:55:22,553 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp/family2/c012deef21c64190a31b6c1d6cd799b4 2023-02-13 00:55:22,553 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp/family2/40cc4bcf9b0e42b18fdb316683d81e79 2023-02-13 00:55:22,557 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp/family1/f84a259856c34b6695a6bb54160a6af0 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/f84a259856c34b6695a6bb54160a6af0 2023-02-13 00:55:22,558 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp/family1/5698639ce41540af9d712c378fe62e38 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/5698639ce41540af9d712c378fe62e38 2023-02-13 00:55:22,561 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/f84a259856c34b6695a6bb54160a6af0, entries=4, sequenceid=5, filesize=5.4 K 2023-02-13 00:55:22,562 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/5698639ce41540af9d712c378fe62e38, entries=4, sequenceid=5, filesize=5.4 K 2023-02-13 00:55:22,562 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp/family2/40cc4bcf9b0e42b18fdb316683d81e79 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/40cc4bcf9b0e42b18fdb316683d81e79 2023-02-13 00:55:22,562 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp/family2/c012deef21c64190a31b6c1d6cd799b4 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/c012deef21c64190a31b6c1d6cd799b4 2023-02-13 00:55:22,566 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/40cc4bcf9b0e42b18fdb316683d81e79, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 00:55:22,567 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/c012deef21c64190a31b6c1d6cd799b4, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 00:55:22,567 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 63f9bed78ea4a19108ec7ecea54182e3 in 1668ms, sequenceid=5, compaction requested=false 2023-02-13 00:55:22,568 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 63f9bed78ea4a19108ec7ecea54182e3: 2023-02-13 00:55:22,568 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:22,568 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 00:55:22,568 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 6b485aea7037b0c8ec399d8612e4ff1d in 1669ms, sequenceid=5, compaction requested=false 2023-02-13 00:55:22,568 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 6b485aea7037b0c8ec399d8612e4ff1d: 2023-02-13 00:55:22,568 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:22,568 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 00:55:22,568 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 00:55:22,568 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:22,568 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-13 00:55:22,568 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-13 00:55:22,570 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:22,570 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:22,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:22,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:22,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:22,570 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:22,571 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:22,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:22,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:22,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:22,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:22,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:22,572 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-13 00:55:22,572 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35ae1d55[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:22,572 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-13 00:55:22,572 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:22,573 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:22,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:22,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:22,573 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-13 00:55:22,574 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-13 00:55:22,574 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:55:22,574 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:22,574 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:22,575 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:22,575 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-13 00:55:22,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:22,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:22,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:22,575 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:22,575 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-13 00:55:22,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:22,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:22,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:22,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:22,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:22,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:22,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:22,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:55:22,578 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-13 00:55:22,578 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-13 00:55:22,578 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:22,578 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:22,578 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-13 00:55:22,578 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:22,580 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:22,580 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:22,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:22,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:22,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:22,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:22,580 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:22,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:22,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:22,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:22,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:22,580 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:22,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:22,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:22,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:22,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:22,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:22,581 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:22,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:22,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:22,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:22,583 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:22,583 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:22,584 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 00:55:22,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:22,584 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-13 00:55:22,584 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:22,584 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-13 00:55:22,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:22,584 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 00:55:22,584 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:22,584 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:22,584 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:22,584 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-13 00:55:22,584 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:22,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:22,584 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:22,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:22,584 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:22,584 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:22,684 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:22,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:22,686 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:55:22,686 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:55:22,687 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:55:22,689 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:22,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-13 00:55:22,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-13 00:55:22,691 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-13 00:55:22,691 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:22,691 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-13 00:55:22,691 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-13 00:55:22,692 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:22,692 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 00:55:22,693 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:22,693 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:22,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:22,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:22,693 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:22,693 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:22,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 00:55:22,694 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:22,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:22,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 00:55:22,694 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-13 00:55:22,694 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-13 00:55:22,694 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-13 00:55:22,694 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-13 00:55:22,694 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:22,694 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-13 00:55:22,695 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:22,695 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. started... 2023-02-13 00:55:22,695 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 00:55:22,695 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:22,695 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:22,695 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 63f9bed78ea4a19108ec7ecea54182e3 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 00:55:22,695 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. started... 2023-02-13 00:55:22,695 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:22,695 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. started... 2023-02-13 00:55:22,695 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 6b485aea7037b0c8ec399d8612e4ff1d 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 00:55:22,695 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 89b71b72a624c785439c7640cd6fb013 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 00:55:22,711 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302133736609080864ed1af2fda92d4e1087b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302133736609080864ed1af2fda92d4e1087b 2023-02-13 00:55:23,110 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021322eec433a1514cdaaa42c42305d30a12 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021322eec433a1514cdaaa42c42305d30a12 2023-02-13 00:55:23,110 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213e74bd61456ae4f18ae9a602ef3d7ea75 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213e74bd61456ae4f18ae9a602ef3d7ea75 2023-02-13 00:55:23,117 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp/family1/54c962e475344ba18eb8b404700e643d 2023-02-13 00:55:23,117 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp/family1/0ed061c653df44ce9f169ddc4b5dfa54 2023-02-13 00:55:23,515 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp/family1/42284400d5f9487b972af1c61186b106 2023-02-13 00:55:23,529 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302131e36bef117a741018eeabfe57867ea24 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302131e36bef117a741018eeabfe57867ea24 2023-02-13 00:55:23,530 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213f7e2ebfd9e7446a4ad37f916104aed78 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213f7e2ebfd9e7446a4ad37f916104aed78 2023-02-13 00:55:23,797 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 00:55:23,797 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x28ef2261 to 127.0.0.1:54478 2023-02-13 00:55:23,797 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:55:23,797 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x188a36eb to 127.0.0.1:54478 2023-02-13 00:55:23,798 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:55:23,798 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 00:55:23,798 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x11ed5c11 to 127.0.0.1:54478 2023-02-13 00:55:23,798 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:55:23,798 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 00:55:23,798 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0c2c4808 to 127.0.0.1:54478 2023-02-13 00:55:23,798 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:55:23,798 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x739763fc to 127.0.0.1:54478 2023-02-13 00:55:23,798 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:55:23,798 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 00:55:23,798 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4b182120 to 127.0.0.1:54478 2023-02-13 00:55:23,798 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:55:23,799 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 00:55:23,799 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0e91d8d3 to 127.0.0.1:54478 2023-02-13 00:55:23,799 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:55:23,929 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302134f35cbbadd6c42a2b8fec88a5e65b1cf to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302134f35cbbadd6c42a2b8fec88a5e65b1cf 2023-02-13 00:55:23,934 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp/family2/d56089120eee43ac9a8f4bee550a86d0 2023-02-13 00:55:23,936 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp/family2/b23fab6f21d54fc79bf2d85aaa21d98d 2023-02-13 00:55:23,938 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp/family1/0ed061c653df44ce9f169ddc4b5dfa54 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/0ed061c653df44ce9f169ddc4b5dfa54 2023-02-13 00:55:23,940 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp/family1/54c962e475344ba18eb8b404700e643d as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/54c962e475344ba18eb8b404700e643d 2023-02-13 00:55:23,941 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/0ed061c653df44ce9f169ddc4b5dfa54, entries=4, sequenceid=9, filesize=5.4 K 2023-02-13 00:55:23,941 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp/family2/d56089120eee43ac9a8f4bee550a86d0 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/d56089120eee43ac9a8f4bee550a86d0 2023-02-13 00:55:23,943 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/54c962e475344ba18eb8b404700e643d, entries=4, sequenceid=9, filesize=5.4 K 2023-02-13 00:55:23,943 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp/family2/b23fab6f21d54fc79bf2d85aaa21d98d as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/b23fab6f21d54fc79bf2d85aaa21d98d 2023-02-13 00:55:23,945 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/d56089120eee43ac9a8f4bee550a86d0, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 00:55:23,946 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 6b485aea7037b0c8ec399d8612e4ff1d in 1251ms, sequenceid=9, compaction requested=false 2023-02-13 00:55:23,946 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 6b485aea7037b0c8ec399d8612e4ff1d: 2023-02-13 00:55:23,946 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:23,946 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 00:55:23,947 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/b23fab6f21d54fc79bf2d85aaa21d98d, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 00:55:23,948 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 63f9bed78ea4a19108ec7ecea54182e3 in 1253ms, sequenceid=9, compaction requested=false 2023-02-13 00:55:23,948 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 63f9bed78ea4a19108ec7ecea54182e3: 2023-02-13 00:55:23,948 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:23,948 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 00:55:23,989 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 00:55:24,334 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp/family2/8b40a4cff14a42e1b4f7b39a7371f146 2023-02-13 00:55:24,338 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp/family1/42284400d5f9487b972af1c61186b106 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/42284400d5f9487b972af1c61186b106 2023-02-13 00:55:24,342 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/42284400d5f9487b972af1c61186b106, entries=4, sequenceid=9, filesize=5.4 K 2023-02-13 00:55:24,343 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp/family2/8b40a4cff14a42e1b4f7b39a7371f146 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/8b40a4cff14a42e1b4f7b39a7371f146 2023-02-13 00:55:24,346 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/8b40a4cff14a42e1b4f7b39a7371f146, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 00:55:24,347 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 89b71b72a624c785439c7640cd6fb013 in 1652ms, sequenceid=9, compaction requested=false 2023-02-13 00:55:24,347 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 89b71b72a624c785439c7640cd6fb013: 2023-02-13 00:55:24,347 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:24,347 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 00:55:24,347 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 00:55:24,347 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:24,347 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-13 00:55:24,347 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-13 00:55:24,349 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:24,349 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:24,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:24,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:24,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:24,350 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:24,350 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:24,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:24,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:24,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:24,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:24,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:24,351 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-13 00:55:24,351 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6817544[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:24,351 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-13 00:55:24,351 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:24,352 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:24,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:24,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:24,352 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-13 00:55:24,352 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-13 00:55:24,352 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:55:24,352 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:24,352 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:24,353 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-13 00:55:24,353 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:24,353 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:24,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:24,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:24,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:24,354 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-13 00:55:24,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:24,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:24,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:24,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:24,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:24,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:24,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:24,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:55:24,356 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-13 00:55:24,356 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-13 00:55:24,356 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:24,356 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:24,356 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-13 00:55:24,356 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:24,357 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:24,357 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:24,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:24,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:24,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:24,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:24,357 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:24,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:24,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:24,358 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:24,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:24,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:24,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:24,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:24,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:24,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:24,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:24,359 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:24,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:24,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:24,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:24,363 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:24,363 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:24,363 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 00:55:24,363 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-13 00:55:24,363 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:24,363 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-13 00:55:24,363 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 00:55:24,363 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:24,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:24,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:24,363 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:24,363 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:24,363 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-13 00:55:24,363 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:24,363 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:24,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:24,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:24,363 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:24,363 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:24,463 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:24,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:24,465 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:55:24,465 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:55:24,465 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:55:24,468 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:24,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-13 00:55:24,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-13 00:55:24,469 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-13 00:55:24,469 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:24,470 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-13 00:55:24,470 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-13 00:55:24,470 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:24,470 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 00:55:24,471 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:24,471 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:24,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:24,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:24,471 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:24,472 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:24,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 00:55:24,472 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:24,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:24,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 00:55:24,472 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-13 00:55:24,473 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-13 00:55:24,473 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-13 00:55:24,473 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-13 00:55:24,473 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:24,473 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-13 00:55:24,473 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:24,473 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 00:55:24,473 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:24,473 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. started... 2023-02-13 00:55:24,473 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:24,473 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:24,473 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. started... 2023-02-13 00:55:24,473 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 89b71b72a624c785439c7640cd6fb013 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 00:55:24,473 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. started... 2023-02-13 00:55:24,473 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 63f9bed78ea4a19108ec7ecea54182e3 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 00:55:24,473 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 6b485aea7037b0c8ec399d8612e4ff1d 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 00:55:24,492 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021308341f081e314a2dae1e3f07ac8a389b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021308341f081e314a2dae1e3f07ac8a389b 2023-02-13 00:55:24,888 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302131dc8db5fc31c4457bc825856a90efcb6 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302131dc8db5fc31c4457bc825856a90efcb6 2023-02-13 00:55:24,892 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213f09c030f2155421aa571ae8f4246407d to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213f09c030f2155421aa571ae8f4246407d 2023-02-13 00:55:24,898 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp/family1/fd70930d268d4ba6afdcdacce99ab021 2023-02-13 00:55:25,292 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp/family1/d0d7911c42c84855967dab4d508097d9 2023-02-13 00:55:25,296 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp/family1/fd053da68487426db3e6dcbdb9604237 2023-02-13 00:55:25,309 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213c108e8256456461b958cb887a71298ec to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213c108e8256456461b958cb887a71298ec 2023-02-13 00:55:25,313 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp/family2/5aa6bed313e74ae0b57bb6aba6db12aa 2023-02-13 00:55:25,317 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp/family1/fd70930d268d4ba6afdcdacce99ab021 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/fd70930d268d4ba6afdcdacce99ab021 2023-02-13 00:55:25,320 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/fd70930d268d4ba6afdcdacce99ab021, entries=4, sequenceid=13, filesize=5.4 K 2023-02-13 00:55:25,321 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp/family2/5aa6bed313e74ae0b57bb6aba6db12aa as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/5aa6bed313e74ae0b57bb6aba6db12aa 2023-02-13 00:55:25,324 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/5aa6bed313e74ae0b57bb6aba6db12aa, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 00:55:25,324 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 89b71b72a624c785439c7640cd6fb013 in 851ms, sequenceid=13, compaction requested=true 2023-02-13 00:55:25,324 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 89b71b72a624c785439c7640cd6fb013: 2023-02-13 00:55:25,325 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:25,325 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 00:55:25,709 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021324cbfacae9a5448894c3419f66a851e0 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023021324cbfacae9a5448894c3419f66a851e0 2023-02-13 00:55:25,711 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213edaa017cbda74487bf56b12844dd7b47 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213edaa017cbda74487bf56b12844dd7b47 2023-02-13 00:55:26,114 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp/family2/9124faf925464c62955abcbcf2b28843 2023-02-13 00:55:26,115 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp/family2/ba1edce2256940b0ab26d5d2235ebf4e 2023-02-13 00:55:26,118 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp/family1/d0d7911c42c84855967dab4d508097d9 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/d0d7911c42c84855967dab4d508097d9 2023-02-13 00:55:26,122 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp/family1/fd053da68487426db3e6dcbdb9604237 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/fd053da68487426db3e6dcbdb9604237 2023-02-13 00:55:26,124 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/d0d7911c42c84855967dab4d508097d9, entries=4, sequenceid=13, filesize=5.4 K 2023-02-13 00:55:26,125 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp/family2/9124faf925464c62955abcbcf2b28843 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/9124faf925464c62955abcbcf2b28843 2023-02-13 00:55:26,125 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/fd053da68487426db3e6dcbdb9604237, entries=4, sequenceid=13, filesize=5.4 K 2023-02-13 00:55:26,126 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp/family2/ba1edce2256940b0ab26d5d2235ebf4e as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/ba1edce2256940b0ab26d5d2235ebf4e 2023-02-13 00:55:26,129 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/9124faf925464c62955abcbcf2b28843, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 00:55:26,130 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 6b485aea7037b0c8ec399d8612e4ff1d in 1657ms, sequenceid=13, compaction requested=true 2023-02-13 00:55:26,130 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 6b485aea7037b0c8ec399d8612e4ff1d: 2023-02-13 00:55:26,130 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:26,130 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 00:55:26,131 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/ba1edce2256940b0ab26d5d2235ebf4e, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 00:55:26,131 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 63f9bed78ea4a19108ec7ecea54182e3 in 1658ms, sequenceid=13, compaction requested=true 2023-02-13 00:55:26,131 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 63f9bed78ea4a19108ec7ecea54182e3: 2023-02-13 00:55:26,131 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:26,131 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 00:55:26,131 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 00:55:26,131 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:26,132 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-13 00:55:26,132 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-13 00:55:26,133 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:26,133 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:26,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:26,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:26,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:26,133 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:26,133 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:26,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:26,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:26,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:26,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:26,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:26,134 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-13 00:55:26,134 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@679de124[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:26,134 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-13 00:55:26,134 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:26,136 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:26,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:26,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:26,136 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:26,136 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:26,136 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-13 00:55:26,136 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-13 00:55:26,136 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:55:26,142 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-13 00:55:26,142 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:26,142 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:26,142 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-13 00:55:26,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:26,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:26,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:26,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:26,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:26,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:26,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:26,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:26,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:26,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:26,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:55:26,144 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-13 00:55:26,144 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-13 00:55:26,144 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:26,144 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:26,144 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-13 00:55:26,144 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:26,146 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:26,146 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:26,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:26,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:26,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:26,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:26,146 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:26,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:26,146 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:26,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:26,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:26,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:26,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:26,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:26,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:26,147 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:26,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:26,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:26,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:26,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:26,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:26,149 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:26,149 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 00:55:26,149 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:26,149 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:26,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:26,149 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:26,149 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-13 00:55:26,150 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:26,149 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 00:55:26,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:26,150 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-13 00:55:26,150 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:26,150 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:26,150 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-13 00:55:26,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:26,150 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:26,150 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:26,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:26,150 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:26,250 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:26,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:26,252 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:55:26,252 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:55:26,252 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:55:26,254 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:26,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-13 00:55:26,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-13 00:55:26,257 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-13 00:55:26,257 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:26,257 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-13 00:55:26,257 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-13 00:55:26,257 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:26,257 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 00:55:26,259 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:26,259 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:26,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:26,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:26,260 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:26,260 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:26,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 00:55:26,260 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:26,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:26,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 00:55:26,261 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-13 00:55:26,261 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-13 00:55:26,261 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-13 00:55:26,261 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-13 00:55:26,261 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:26,261 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-13 00:55:26,261 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:26,261 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. started... 2023-02-13 00:55:26,261 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 63f9bed78ea4a19108ec7ecea54182e3 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 00:55:26,262 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:26,262 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 00:55:26,262 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:26,262 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. started... 2023-02-13 00:55:26,262 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:26,262 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. started... 2023-02-13 00:55:26,262 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 6b485aea7037b0c8ec399d8612e4ff1d 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 00:55:26,262 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 89b71b72a624c785439c7640cd6fb013 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 00:55:26,286 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302138f21581a0c4c4b58a0ad086df3e0ae67 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302138f21581a0c4c4b58a0ad086df3e0ae67 2023-02-13 00:55:26,287 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302132917a3ddef6a4d7e847180bde371f97b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302132917a3ddef6a4d7e847180bde371f97b 2023-02-13 00:55:26,298 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp/family1/04385aa81e7440e7a4d326422c60873f 2023-02-13 00:55:26,311 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213e50db5b4ac5543ffa6bffd3c50af6216 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213e50db5b4ac5543ffa6bffd3c50af6216 2023-02-13 00:55:26,679 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213c7c589f59d5a46e69f929bfe1462e530 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213c7c589f59d5a46e69f929bfe1462e530 2023-02-13 00:55:26,699 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp/family1/85892d97287240ffb900a29077361472 2023-02-13 00:55:26,715 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp/family2/b136ea849d0e4e6a8cbe2bb45e6ac4b8 2023-02-13 00:55:26,719 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp/family1/04385aa81e7440e7a4d326422c60873f as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/04385aa81e7440e7a4d326422c60873f 2023-02-13 00:55:26,722 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/04385aa81e7440e7a4d326422c60873f, entries=4, sequenceid=17, filesize=5.4 K 2023-02-13 00:55:26,723 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp/family2/b136ea849d0e4e6a8cbe2bb45e6ac4b8 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/b136ea849d0e4e6a8cbe2bb45e6ac4b8 2023-02-13 00:55:26,726 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/b136ea849d0e4e6a8cbe2bb45e6ac4b8, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 00:55:26,727 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 89b71b72a624c785439c7640cd6fb013 in 465ms, sequenceid=17, compaction requested=true 2023-02-13 00:55:26,727 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 89b71b72a624c785439c7640cd6fb013: 2023-02-13 00:55:26,727 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:26,727 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 00:55:27,084 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp/family1/61f19ef16aa843998ec2e2f36c298aa1 2023-02-13 00:55:27,110 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021321d6790068c3494193aa9a5eb201e991 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023021321d6790068c3494193aa9a5eb201e991 2023-02-13 00:55:27,497 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213d3021403671e42d48df383dfdac034be to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213d3021403671e42d48df383dfdac034be 2023-02-13 00:55:27,514 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp/family2/d095b833edb04e21a0f6de413b1e43f9 2023-02-13 00:55:27,518 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp/family1/85892d97287240ffb900a29077361472 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/85892d97287240ffb900a29077361472 2023-02-13 00:55:27,521 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/85892d97287240ffb900a29077361472, entries=4, sequenceid=17, filesize=5.4 K 2023-02-13 00:55:27,522 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp/family2/d095b833edb04e21a0f6de413b1e43f9 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/d095b833edb04e21a0f6de413b1e43f9 2023-02-13 00:55:27,525 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/d095b833edb04e21a0f6de413b1e43f9, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 00:55:27,526 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 6b485aea7037b0c8ec399d8612e4ff1d in 1264ms, sequenceid=17, compaction requested=true 2023-02-13 00:55:27,526 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 6b485aea7037b0c8ec399d8612e4ff1d: 2023-02-13 00:55:27,526 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:27,526 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 00:55:27,902 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp/family2/ad7c82764fe141a49defe528f6bd7f78 2023-02-13 00:55:27,906 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp/family1/61f19ef16aa843998ec2e2f36c298aa1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/61f19ef16aa843998ec2e2f36c298aa1 2023-02-13 00:55:27,908 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/61f19ef16aa843998ec2e2f36c298aa1, entries=4, sequenceid=17, filesize=5.4 K 2023-02-13 00:55:27,909 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp/family2/ad7c82764fe141a49defe528f6bd7f78 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/ad7c82764fe141a49defe528f6bd7f78 2023-02-13 00:55:27,912 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/ad7c82764fe141a49defe528f6bd7f78, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 00:55:27,913 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 63f9bed78ea4a19108ec7ecea54182e3 in 1652ms, sequenceid=17, compaction requested=true 2023-02-13 00:55:27,913 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 63f9bed78ea4a19108ec7ecea54182e3: 2023-02-13 00:55:27,913 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:27,913 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 00:55:27,913 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 00:55:27,913 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:27,913 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-13 00:55:27,913 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-13 00:55:27,915 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:27,915 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:27,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:27,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:27,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:27,915 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:27,915 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:27,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:27,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:27,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:27,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:27,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:27,916 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-13 00:55:27,916 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2e85ba5e[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:27,916 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-13 00:55:27,916 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:27,917 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:27,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:27,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:27,917 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-13 00:55:27,917 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-13 00:55:27,917 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:27,917 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:27,917 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:55:27,919 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-13 00:55:27,919 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:27,919 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:27,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:27,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:27,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:27,919 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-13 00:55:27,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:27,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:27,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:27,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:27,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:27,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:27,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:27,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:55:27,921 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-13 00:55:27,921 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:27,921 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:27,921 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-13 00:55:27,921 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-13 00:55:27,921 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:27,923 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:27,923 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:27,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:27,924 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:27,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:27,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:27,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:27,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:27,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:27,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:27,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:27,924 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:27,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:27,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:27,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:27,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:27,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:27,925 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:27,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:27,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:27,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:27,927 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:27,927 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:27,927 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 00:55:27,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:27,927 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:27,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:27,927 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-13 00:55:27,927 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:27,927 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 00:55:27,927 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:27,927 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:27,927 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:27,927 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:27,927 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:27,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:27,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:27,927 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-13 00:55:27,927 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:28,028 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:28,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:28,038 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=snaptb-1676249728028 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-13 00:55:28,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1676249728038 (current time:1676249728038). 2023-02-13 00:55:28,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-02-13 00:55:28,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] zookeeper.ReadOnlyZKClient(139): Connect 0x1b29fe8f to 127.0.0.1:54478 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:55:28,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2cdcb7d1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:55:28,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:55:28,053 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44390, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:55:28,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1b29fe8f to 127.0.0.1:54478 2023-02-13 00:55:28,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:55:28,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-02-13 00:55:28,055 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=snaptb-1676249728028 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-13 00:55:28,062 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure2.ProcedureExecutor(1029): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-13 00:55:28,063 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-13 00:55:28,063 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1676250328063 2023-02-13 00:55:28,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] snapshot.SnapshotManager(653): Started snapshot: { ss=snaptb-1676249728028 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-13 00:55:28,064 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1676249728028 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-02-13 00:55:28,065 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-13 00:55:28,065 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1029): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-13 00:55:28,066 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 3 msec 2023-02-13 00:55:28,066 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-13 00:55:28,067 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1676250328067 2023-02-13 00:55:28,069 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1676249728028 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-02-13 00:55:28,069 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-02-13 00:55:28,170 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 00:55:28,172 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676249728028 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 00:55:28,173 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1676249728028 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 00:55:28,173 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-02-13 00:55:28,373 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 00:55:28,374 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676249728028 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 00:55:28,374 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1676249728028 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 00:55:28,374 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-02-13 00:55:28,475 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1676249728028 2023-02-13 00:55:28,476 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1676249728028' 2023-02-13 00:55:28,476 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-02-13 00:55:28,476 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1676249728028' starting 'acquire' 2023-02-13 00:55:28,476 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1676249728028', kicking off acquire phase on members. 2023-02-13 00:55:28,476 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1676249728028 2023-02-13 00:55:28,476 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1676249728028 2023-02-13 00:55:28,477 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-02-13 00:55:28,477 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1676249728028/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:28,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-02-13 00:55:28,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 00:55:28,478 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1676249728028/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:28,478 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:28,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1676249728028 2023-02-13 00:55:28,478 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1676249728028 2023-02-13 00:55:28,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-02-13 00:55:28,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1676249728028 2023-02-13 00:55:28,478 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1676249728028 from table testCompactionWithHFileLink type FLUSH 2023-02-13 00:55:28,480 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1676249728028 2023-02-13 00:55:28,481 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1676249728028' with timeout 300000ms 2023-02-13 00:55:28,481 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-02-13 00:55:28,481 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1676249728028' starting 'acquire' stage 2023-02-13 00:55:28,481 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1676249728028' locally acquired 2023-02-13 00:55:28,481 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (snaptb-1676249728028) in zk 2023-02-13 00:55:28,482 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1676249728028/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:28,482 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1676249728028 2023-02-13 00:55:28,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1676249728028/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:28,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:28,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-13 00:55:28,482 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1676249728028 2023-02-13 00:55:28,482 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1676249728028' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:28,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:28,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:28,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676249728028 2023-02-13 00:55:28,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:28,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:28,484 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'snaptb-1676249728028' on coordinator 2023-02-13 00:55:28,484 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6b045e6e[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:28,484 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1676249728028' starting 'in-barrier' execution. 2023-02-13 00:55:28,484 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1676249728028 2023-02-13 00:55:28,488 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1676249728028 2023-02-13 00:55:28,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1676249728028 2023-02-13 00:55:28,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1676249728028 2023-02-13 00:55:28,488 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1676249728028' received 'reached' from coordinator. 2023-02-13 00:55:28,488 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1676249728028/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:28,488 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:28,489 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:28,489 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-02-13 00:55:28,489 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:28,489 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-02-13 00:55:28,489 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:28,489 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. started... 2023-02-13 00:55:28,489 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. started... 2023-02-13 00:55:28,489 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. started... 2023-02-13 00:55:28,490 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-0] regionserver.HRegion(2445): Flush status journal for 63f9bed78ea4a19108ec7ecea54182e3: 2023-02-13 00:55:28,490 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-2] regionserver.HRegion(2445): Flush status journal for 89b71b72a624c785439c7640cd6fb013: 2023-02-13 00:55:28,490 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-1] regionserver.HRegion(2445): Flush status journal for 6b485aea7037b0c8ec399d8612e4ff1d: 2023-02-13 00:55:28,490 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3.' region-info for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,490 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013.' region-info for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,490 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d.' region-info for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,495 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-13 00:55:28,495 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-13 00:55:28,495 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-13 00:55:28,635 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-02-13 00:55:28,638 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/f84a259856c34b6695a6bb54160a6af0, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/54c962e475344ba18eb8b404700e643d, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/fd053da68487426db3e6dcbdb9604237, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/61f19ef16aa843998ec2e2f36c298aa1] hfiles 2023-02-13 00:55:28,638 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/5698639ce41540af9d712c378fe62e38, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/0ed061c653df44ce9f169ddc4b5dfa54, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/d0d7911c42c84855967dab4d508097d9, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/85892d97287240ffb900a29077361472] hfiles 2023-02-13 00:55:28,638 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/b764582729ff4ae3a7b761976a0a3c41, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/42284400d5f9487b972af1c61186b106, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/fd70930d268d4ba6afdcdacce99ab021, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/04385aa81e7440e7a4d326422c60873f] hfiles 2023-02-13 00:55:28,638 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/5698639ce41540af9d712c378fe62e38 for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,638 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/f84a259856c34b6695a6bb54160a6af0 for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,638 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/b764582729ff4ae3a7b761976a0a3c41 for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,641 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/54c962e475344ba18eb8b404700e643d for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,641 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/42284400d5f9487b972af1c61186b106 for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,641 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/0ed061c653df44ce9f169ddc4b5dfa54 for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,641 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/fd70930d268d4ba6afdcdacce99ab021 for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,641 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/fd053da68487426db3e6dcbdb9604237 for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,642 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/04385aa81e7440e7a4d326422c60873f for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,642 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/d0d7911c42c84855967dab4d508097d9 for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,642 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/61f19ef16aa843998ec2e2f36c298aa1 for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,642 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/85892d97287240ffb900a29077361472 for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,642 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/a0f7d35e413a49d6bf0acc6f8d6693b6, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/8b40a4cff14a42e1b4f7b39a7371f146, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/5aa6bed313e74ae0b57bb6aba6db12aa, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/b136ea849d0e4e6a8cbe2bb45e6ac4b8] hfiles 2023-02-13 00:55:28,642 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/c012deef21c64190a31b6c1d6cd799b4, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/d56089120eee43ac9a8f4bee550a86d0, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/9124faf925464c62955abcbcf2b28843, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/d095b833edb04e21a0f6de413b1e43f9] hfiles 2023-02-13 00:55:28,642 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/c012deef21c64190a31b6c1d6cd799b4 for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,642 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/a0f7d35e413a49d6bf0acc6f8d6693b6 for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,642 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/40cc4bcf9b0e42b18fdb316683d81e79, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/b23fab6f21d54fc79bf2d85aaa21d98d, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/ba1edce2256940b0ab26d5d2235ebf4e, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/ad7c82764fe141a49defe528f6bd7f78] hfiles 2023-02-13 00:55:28,642 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/8b40a4cff14a42e1b4f7b39a7371f146 for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,642 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/40cc4bcf9b0e42b18fdb316683d81e79 for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,642 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/d56089120eee43ac9a8f4bee550a86d0 for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,642 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/b23fab6f21d54fc79bf2d85aaa21d98d for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,642 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/9124faf925464c62955abcbcf2b28843 for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,642 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/ba1edce2256940b0ab26d5d2235ebf4e for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,642 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/d095b833edb04e21a0f6de413b1e43f9 for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,642 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/5aa6bed313e74ae0b57bb6aba6db12aa for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,642 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/ad7c82764fe141a49defe528f6bd7f78 for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,642 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/b136ea849d0e4e6a8cbe2bb45e6ac4b8 for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,650 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. completed. 2023-02-13 00:55:28,650 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. completed. 2023-02-13 00:55:28,650 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:28,650 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. completed. 2023-02-13 00:55:28,650 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:28,650 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:28,651 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-02-13 00:55:28,651 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-02-13 00:55:28,651 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-02-13 00:55:28,651 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-02-13 00:55:28,651 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:28,651 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1676249728028' locally completed 2023-02-13 00:55:28,651 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1676249728028' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:55:28,653 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1676249728028/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:28,653 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1676249728028' has notified controller of completion 2023-02-13 00:55:28,653 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:28,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1676249728028/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:28,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:28,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-13 00:55:28,653 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1676249728028' completed. 2023-02-13 00:55:28,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:28,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:28,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676249728028 2023-02-13 00:55:28,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:28,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:28,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676249728028 2023-02-13 00:55:28,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:28,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1676249728028' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:55:28,655 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'snaptb-1676249728028', counting down latch. Waiting for 0 more 2023-02-13 00:55:28,655 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1676249728028' execution completed 2023-02-13 00:55:28,655 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:28,655 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:28,655 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1676249728028 2023-02-13 00:55:28,655 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1676249728028including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-02-13 00:55:28,656 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1676249728028 2023-02-13 00:55:28,656 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1676249728028 2023-02-13 00:55:28,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1676249728028 2023-02-13 00:55:28,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1676249728028 2023-02-13 00:55:28,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:28,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-13 00:55:28,656 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-02-13 00:55:28,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1676249728028 2023-02-13 00:55:28,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:28,657 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1676249728028/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:28,657 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-02-13 00:55:28,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 00:55:28,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676249728028 2023-02-13 00:55:28,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1676249728028 2023-02-13 00:55:28,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:28,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676249728028 2023-02-13 00:55:28,658 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1676249728028/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:28,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:28,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:28,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676249728028 2023-02-13 00:55:28,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:28,660 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-02-13 00:55:28,660 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1676249728028/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:28,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-02-13 00:55:28,660 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:28,660 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1676249728028 2023-02-13 00:55:28,660 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-02-13 00:55:28,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 00:55:28,660 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1676249728028 2023-02-13 00:55:28,660 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-02-13 00:55:28,661 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1676249728028 2023-02-13 00:55:28,661 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1676249728028/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:28,661 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1676249728028 2023-02-13 00:55:28,661 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1676249728028 2023-02-13 00:55:28,661 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1676249728028 2023-02-13 00:55:28,661 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-02-13 00:55:28,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-02-13 00:55:28,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 00:55:28,661 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1676249728028 2023-02-13 00:55:28,662 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302131dc8db5fc31c4457bc825856a90efcb6, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302132917a3ddef6a4d7e847180bde371f97b, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302133736609080864ed1af2fda92d4e1087b, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302139ab092340bd14dbc8b723519529420ae, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302136a074e5589c4444cbea4ec51c8c112f7, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213c7c589f59d5a46e69f929bfe1462e530, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213e74bd61456ae4f18ae9a602ef3d7ea75, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213f09c030f2155421aa571ae8f4246407d, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021308341f081e314a2dae1e3f07ac8a389b, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021322eec433a1514cdaaa42c42305d30a12, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213348ca6e4353c43259c1482c61f82b4a5, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302138f21581a0c4c4b58a0ad086df3e0ae67] mob files 2023-02-13 00:55:28,662 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302131dc8db5fc31c4457bc825856a90efcb6 2023-02-13 00:55:28,662 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302132917a3ddef6a4d7e847180bde371f97b 2023-02-13 00:55:28,662 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302133736609080864ed1af2fda92d4e1087b 2023-02-13 00:55:28,662 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302139ab092340bd14dbc8b723519529420ae 2023-02-13 00:55:28,662 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302136a074e5589c4444cbea4ec51c8c112f7 2023-02-13 00:55:28,662 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213c7c589f59d5a46e69f929bfe1462e530 2023-02-13 00:55:28,662 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213e74bd61456ae4f18ae9a602ef3d7ea75 2023-02-13 00:55:28,662 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213f09c030f2155421aa571ae8f4246407d 2023-02-13 00:55:28,662 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021308341f081e314a2dae1e3f07ac8a389b 2023-02-13 00:55:28,662 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021322eec433a1514cdaaa42c42305d30a12 2023-02-13 00:55:28,662 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213348ca6e4353c43259c1482c61f82b4a5 2023-02-13 00:55:28,662 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302138f21581a0c4c4b58a0ad086df3e0ae67 2023-02-13 00:55:28,663 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302131e36bef117a741018eeabfe57867ea24, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023021321d6790068c3494193aa9a5eb201e991, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023021324cbfacae9a5448894c3419f66a851e0, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302134f08f76cbaf1432b9c2e01f3aeffa05c, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023021329cbdffeaba74431b13e6040dd8db724, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213d3021403671e42d48df383dfdac034be, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213edaa017cbda74487bf56b12844dd7b47, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213f7e2ebfd9e7446a4ad37f916104aed78, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023021301e50cc3adca4f52a6f0a591b7de5f77, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302134f35cbbadd6c42a2b8fec88a5e65b1cf, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213c108e8256456461b958cb887a71298ec, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213e50db5b4ac5543ffa6bffd3c50af6216] mob files 2023-02-13 00:55:28,663 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302131e36bef117a741018eeabfe57867ea24 2023-02-13 00:55:28,663 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023021321d6790068c3494193aa9a5eb201e991 2023-02-13 00:55:28,663 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023021324cbfacae9a5448894c3419f66a851e0 2023-02-13 00:55:28,663 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302134f08f76cbaf1432b9c2e01f3aeffa05c 2023-02-13 00:55:28,663 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023021329cbdffeaba74431b13e6040dd8db724 2023-02-13 00:55:28,663 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213d3021403671e42d48df383dfdac034be 2023-02-13 00:55:28,664 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213edaa017cbda74487bf56b12844dd7b47 2023-02-13 00:55:28,664 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213f7e2ebfd9e7446a4ad37f916104aed78 2023-02-13 00:55:28,664 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023021301e50cc3adca4f52a6f0a591b7de5f77 2023-02-13 00:55:28,664 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302134f35cbbadd6c42a2b8fec88a5e65b1cf 2023-02-13 00:55:28,664 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213c108e8256456461b958cb887a71298ec 2023-02-13 00:55:28,664 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213e50db5b4ac5543ffa6bffd3c50af6216 2023-02-13 00:55:28,674 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 00:55:28,675 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676249728028 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 00:55:28,675 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1676249728028 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 00:55:28,676 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-02-13 00:55:29,068 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1676249728028 2023-02-13 00:55:29,070 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.hbase-snapshot/.tmp/snaptb-1676249728028 2023-02-13 00:55:29,176 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 00:55:29,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676249728028 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 00:55:29,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1676249728028 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 00:55:29,177 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-02-13 00:55:29,515 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.hbase-snapshot/.tmp/snaptb-1676249728028 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.hbase-snapshot/snaptb-1676249728028 2023-02-13 00:55:30,177 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 00:55:30,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676249728028 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 00:55:30,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1676249728028 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 00:55:30,178 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2428): (#6) Sleeping: 2000ms while waiting for snapshot completion. 2023-02-13 00:55:30,330 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1676249728028 of table testCompactionWithHFileLink completed 2023-02-13 00:55:30,330 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.hbase-snapshot/.tmp/snaptb-1676249728028 2023-02-13 00:55:30,331 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.hbase-snapshot/.tmp/snaptb-1676249728028 2023-02-13 00:55:30,331 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1676249728028 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1676249728064Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1676249729068 (+1004 ms)Consolidate snapshot: snaptb-1676249728028 at 1676249729068Loading Region manifests for snaptb-1676249728028 at 1676249729069 (+1 ms)Writing data manifest for snaptb-1676249728028 at 1676249729079 (+10 ms)Verifying snapshot: snaptb-1676249728028 at 1676249729486 (+407 ms)Snapshot snaptb-1676249728028 of table testCompactionWithHFileLink completed at 1676249730330 (+844 ms) 2023-02-13 00:55:30,332 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-13 00:55:30,333 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 2.2670 sec 2023-02-13 00:55:32,178 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 00:55:32,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676249728028 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 00:55:32,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] snapshot.SnapshotManager(401): Snapshot '{ ss=snaptb-1676249728028 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-02-13 00:55:32,208 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:32,209 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-13 00:55:32,209 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-13 00:55:32,209 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-13 00:55:32,209 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:32,209 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-13 00:55:32,209 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-13 00:55:32,210 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:32,210 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 00:55:32,212 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:32,212 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:32,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:32,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:32,212 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:32,212 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:32,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 00:55:32,213 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:32,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:32,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 00:55:32,213 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-13 00:55:32,213 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-13 00:55:32,213 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-13 00:55:32,213 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:32,213 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-13 00:55:32,214 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:32,214 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. started... 2023-02-13 00:55:32,214 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 63f9bed78ea4a19108ec7ecea54182e3 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 00:55:32,215 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:32,215 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 00:55:32,215 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:32,215 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:32,215 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. started... 2023-02-13 00:55:32,215 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. started... 2023-02-13 00:55:32,215 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 89b71b72a624c785439c7640cd6fb013 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 00:55:32,215 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 6b485aea7037b0c8ec399d8612e4ff1d 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 00:55:32,627 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp/family1/fb94a170d3444262b51b05c3598b0642 2023-02-13 00:55:32,627 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp/family1/05c6a7b6fdaa414ab8db4a3c7f83b746 2023-02-13 00:55:32,628 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp/family1/38d794b77afe4caebf63da235971e4d4 2023-02-13 00:55:32,631 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb94a170d3444262b51b05c3598b0642 2023-02-13 00:55:32,631 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 38d794b77afe4caebf63da235971e4d4 2023-02-13 00:55:32,632 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 05c6a7b6fdaa414ab8db4a3c7f83b746 2023-02-13 00:55:33,444 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp/family2/a4164f603b5c467181f95a60bd292b8b 2023-02-13 00:55:33,445 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp/family2/bd0d461eb72541228181e06fef48cf9c 2023-02-13 00:55:33,445 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp/family2/671b924f27c74cc7918fc49264df3b33 2023-02-13 00:55:33,448 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 671b924f27c74cc7918fc49264df3b33 2023-02-13 00:55:33,448 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bd0d461eb72541228181e06fef48cf9c 2023-02-13 00:55:33,449 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a4164f603b5c467181f95a60bd292b8b 2023-02-13 00:55:33,449 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp/family1/05c6a7b6fdaa414ab8db4a3c7f83b746 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/05c6a7b6fdaa414ab8db4a3c7f83b746 2023-02-13 00:55:33,449 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp/family1/fb94a170d3444262b51b05c3598b0642 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/fb94a170d3444262b51b05c3598b0642 2023-02-13 00:55:33,449 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp/family1/38d794b77afe4caebf63da235971e4d4 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/38d794b77afe4caebf63da235971e4d4 2023-02-13 00:55:33,453 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 05c6a7b6fdaa414ab8db4a3c7f83b746 2023-02-13 00:55:33,454 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 38d794b77afe4caebf63da235971e4d4 2023-02-13 00:55:33,454 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/38d794b77afe4caebf63da235971e4d4, entries=3, sequenceid=24, filesize=5.0 K 2023-02-13 00:55:33,454 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/05c6a7b6fdaa414ab8db4a3c7f83b746, entries=3, sequenceid=24, filesize=5.0 K 2023-02-13 00:55:33,454 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb94a170d3444262b51b05c3598b0642 2023-02-13 00:55:33,454 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/fb94a170d3444262b51b05c3598b0642, entries=3, sequenceid=24, filesize=5.0 K 2023-02-13 00:55:33,455 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp/family2/bd0d461eb72541228181e06fef48cf9c as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/bd0d461eb72541228181e06fef48cf9c 2023-02-13 00:55:33,455 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp/family2/671b924f27c74cc7918fc49264df3b33 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/671b924f27c74cc7918fc49264df3b33 2023-02-13 00:55:33,455 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp/family2/a4164f603b5c467181f95a60bd292b8b as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/a4164f603b5c467181f95a60bd292b8b 2023-02-13 00:55:33,459 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bd0d461eb72541228181e06fef48cf9c 2023-02-13 00:55:33,460 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a4164f603b5c467181f95a60bd292b8b 2023-02-13 00:55:33,460 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/bd0d461eb72541228181e06fef48cf9c, entries=1, sequenceid=24, filesize=5.0 K 2023-02-13 00:55:33,460 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 671b924f27c74cc7918fc49264df3b33 2023-02-13 00:55:33,460 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/a4164f603b5c467181f95a60bd292b8b, entries=1, sequenceid=24, filesize=5.0 K 2023-02-13 00:55:33,460 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/671b924f27c74cc7918fc49264df3b33, entries=1, sequenceid=24, filesize=5.0 K 2023-02-13 00:55:33,461 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 63f9bed78ea4a19108ec7ecea54182e3 in 1246ms, sequenceid=24, compaction requested=true 2023-02-13 00:55:33,461 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 6b485aea7037b0c8ec399d8612e4ff1d in 1246ms, sequenceid=24, compaction requested=true 2023-02-13 00:55:33,461 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 63f9bed78ea4a19108ec7ecea54182e3: 2023-02-13 00:55:33,461 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 89b71b72a624c785439c7640cd6fb013 in 1246ms, sequenceid=24, compaction requested=true 2023-02-13 00:55:33,461 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:33,461 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 6b485aea7037b0c8ec399d8612e4ff1d: 2023-02-13 00:55:33,461 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 00:55:33,461 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 89b71b72a624c785439c7640cd6fb013: 2023-02-13 00:55:33,461 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:33,461 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:33,461 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 00:55:33,461 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 00:55:33,461 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 00:55:33,461 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:33,461 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-13 00:55:33,461 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-13 00:55:33,463 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:33,463 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:33,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:33,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:33,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:33,463 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:33,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:33,463 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:33,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:33,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:33,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:33,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:33,465 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-13 00:55:33,465 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-13 00:55:33,465 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@66222d1f[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:33,465 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:33,472 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:33,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:33,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:33,472 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-13 00:55:33,473 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-13 00:55:33,472 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:33,473 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:33,473 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:55:33,474 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-13 00:55:33,474 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:33,474 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:33,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:33,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:33,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:33,474 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-13 00:55:33,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:33,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:33,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:33,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:33,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:33,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:33,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:33,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:55:33,476 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-13 00:55:33,476 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-13 00:55:33,476 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:33,476 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:33,476 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-13 00:55:33,476 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:33,478 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:33,478 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:33,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:33,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:33,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:33,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:33,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:33,478 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:33,478 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:33,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:33,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:33,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:33,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:33,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:33,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:33,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:33,479 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:33,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:33,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:33,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 00:55:33,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:33,483 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:33,483 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 00:55:33,483 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 00:55:33,483 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:33,483 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:33,483 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 00:55:33,483 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:33,483 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 00:55:33,483 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:33,483 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-13 00:55:33,483 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:33,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:33,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:33,483 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:33,484 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-13 00:55:33,484 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-13 00:55:33,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:33,484 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:33,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:33,584 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:33,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:33,585 DEBUG [Listener at localhost/37869] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 00:55:33,586 DEBUG [Listener at localhost/37869] regionserver.HStore(1912): 63f9bed78ea4a19108ec7ecea54182e3/family1 is initiating major compaction (all files) 2023-02-13 00:55:33,586 INFO [Listener at localhost/37869] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 00:55:33,586 INFO [Listener at localhost/37869] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:55:33,586 INFO [Listener at localhost/37869] regionserver.HRegion(2258): Starting compaction of 63f9bed78ea4a19108ec7ecea54182e3/family1 in testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:33,586 INFO [Listener at localhost/37869] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/f84a259856c34b6695a6bb54160a6af0, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/54c962e475344ba18eb8b404700e643d, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/fd053da68487426db3e6dcbdb9604237, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/61f19ef16aa843998ec2e2f36c298aa1, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/38d794b77afe4caebf63da235971e4d4] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp, totalSize=26.7 K 2023-02-13 00:55:33,587 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting f84a259856c34b6695a6bb54160a6af0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676249720885 2023-02-13 00:55:33,587 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 54c962e475344ba18eb8b404700e643d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676249722687 2023-02-13 00:55:33,587 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting fd053da68487426db3e6dcbdb9604237, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676249724465 2023-02-13 00:55:33,588 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 61f19ef16aa843998ec2e2f36c298aa1, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676249726252 2023-02-13 00:55:33,588 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 38d794b77afe4caebf63da235971e4d4, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-13 00:55:33,601 INFO [Listener at localhost/37869] throttle.PressureAwareThroughputController(145): 63f9bed78ea4a19108ec7ecea54182e3#family1#compaction#107 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:55:34,009 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023021309328d5dd03c45fa9f52ce41e3e16351_del 2023-02-13 00:55:34,009 INFO [Listener at localhost/37869] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021309328d5dd03c45fa9f52ce41e3e16351_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021309328d5dd03c45fa9f52ce41e3e16351_del 2023-02-13 00:55:34,822 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3a504ca96acf4c6a9a3dbc93c6381047 2023-02-13 00:55:34,823 DEBUG [Listener at localhost/37869] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp/family1/3a504ca96acf4c6a9a3dbc93c6381047 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/3a504ca96acf4c6a9a3dbc93c6381047 2023-02-13 00:55:34,826 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3a504ca96acf4c6a9a3dbc93c6381047 2023-02-13 00:55:34,827 INFO [Listener at localhost/37869] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 63f9bed78ea4a19108ec7ecea54182e3/family1 of 63f9bed78ea4a19108ec7ecea54182e3 into 3a504ca96acf4c6a9a3dbc93c6381047(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 00:55:34,827 DEBUG [Listener at localhost/37869] regionserver.HRegion(2288): Compaction status journal for 63f9bed78ea4a19108ec7ecea54182e3: 2023-02-13 00:55:34,827 DEBUG [Listener at localhost/37869] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 00:55:34,827 DEBUG [Listener at localhost/37869] regionserver.HStore(1912): 63f9bed78ea4a19108ec7ecea54182e3/family2 is initiating major compaction (all files) 2023-02-13 00:55:34,827 INFO [Listener at localhost/37869] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 00:55:34,827 INFO [Listener at localhost/37869] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:55:34,828 INFO [Listener at localhost/37869] regionserver.HRegion(2258): Starting compaction of 63f9bed78ea4a19108ec7ecea54182e3/family2 in testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:34,828 INFO [Listener at localhost/37869] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/40cc4bcf9b0e42b18fdb316683d81e79, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/b23fab6f21d54fc79bf2d85aaa21d98d, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/ba1edce2256940b0ab26d5d2235ebf4e, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/ad7c82764fe141a49defe528f6bd7f78, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/bd0d461eb72541228181e06fef48cf9c] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp, totalSize=25.4 K 2023-02-13 00:55:34,828 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 40cc4bcf9b0e42b18fdb316683d81e79, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676249720885 2023-02-13 00:55:34,828 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting b23fab6f21d54fc79bf2d85aaa21d98d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676249722687 2023-02-13 00:55:34,829 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting ba1edce2256940b0ab26d5d2235ebf4e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676249724465 2023-02-13 00:55:34,829 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting ad7c82764fe141a49defe528f6bd7f78, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676249726252 2023-02-13 00:55:34,829 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting bd0d461eb72541228181e06fef48cf9c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-13 00:55:34,836 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/f84a259856c34b6695a6bb54160a6af0, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/54c962e475344ba18eb8b404700e643d, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/fd053da68487426db3e6dcbdb9604237, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/61f19ef16aa843998ec2e2f36c298aa1, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/38d794b77afe4caebf63da235971e4d4] to archive 2023-02-13 00:55:34,836 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:34,838 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/f84a259856c34b6695a6bb54160a6af0 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/f84a259856c34b6695a6bb54160a6af0 2023-02-13 00:55:34,839 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/54c962e475344ba18eb8b404700e643d to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/54c962e475344ba18eb8b404700e643d 2023-02-13 00:55:34,839 INFO [Listener at localhost/37869] throttle.PressureAwareThroughputController(145): 63f9bed78ea4a19108ec7ecea54182e3#family2#compaction#108 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:55:34,840 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/fd053da68487426db3e6dcbdb9604237 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/fd053da68487426db3e6dcbdb9604237 2023-02-13 00:55:34,841 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/61f19ef16aa843998ec2e2f36c298aa1 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/61f19ef16aa843998ec2e2f36c298aa1 2023-02-13 00:55:34,842 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/38d794b77afe4caebf63da235971e4d4 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/38d794b77afe4caebf63da235971e4d4 2023-02-13 00:55:35,248 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230213fb3cf86ef18546138ec770b18d54c288_del 2023-02-13 00:55:35,248 INFO [Listener at localhost/37869] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213fb3cf86ef18546138ec770b18d54c288_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213fb3cf86ef18546138ec770b18d54c288_del 2023-02-13 00:55:36,061 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 01efca4802b34dc8baf2986ca31e146b 2023-02-13 00:55:36,062 DEBUG [Listener at localhost/37869] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp/family2/01efca4802b34dc8baf2986ca31e146b as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/01efca4802b34dc8baf2986ca31e146b 2023-02-13 00:55:36,065 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 01efca4802b34dc8baf2986ca31e146b 2023-02-13 00:55:36,066 INFO [Listener at localhost/37869] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 63f9bed78ea4a19108ec7ecea54182e3/family2 of 63f9bed78ea4a19108ec7ecea54182e3 into 01efca4802b34dc8baf2986ca31e146b(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 00:55:36,066 DEBUG [Listener at localhost/37869] regionserver.HRegion(2288): Compaction status journal for 63f9bed78ea4a19108ec7ecea54182e3: 2023-02-13 00:55:36,066 DEBUG [Listener at localhost/37869] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 00:55:36,066 DEBUG [Listener at localhost/37869] regionserver.HStore(1912): 6b485aea7037b0c8ec399d8612e4ff1d/family1 is initiating major compaction (all files) 2023-02-13 00:55:36,066 INFO [Listener at localhost/37869] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 00:55:36,066 INFO [Listener at localhost/37869] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:55:36,066 INFO [Listener at localhost/37869] regionserver.HRegion(2258): Starting compaction of 6b485aea7037b0c8ec399d8612e4ff1d/family1 in testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:36,067 INFO [Listener at localhost/37869] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/5698639ce41540af9d712c378fe62e38, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/0ed061c653df44ce9f169ddc4b5dfa54, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/d0d7911c42c84855967dab4d508097d9, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/85892d97287240ffb900a29077361472, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/05c6a7b6fdaa414ab8db4a3c7f83b746] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp, totalSize=26.7 K 2023-02-13 00:55:36,067 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 5698639ce41540af9d712c378fe62e38, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676249720884 2023-02-13 00:55:36,067 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 0ed061c653df44ce9f169ddc4b5dfa54, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676249722686 2023-02-13 00:55:36,068 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting d0d7911c42c84855967dab4d508097d9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676249724465 2023-02-13 00:55:36,068 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 85892d97287240ffb900a29077361472, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676249726252 2023-02-13 00:55:36,068 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 05c6a7b6fdaa414ab8db4a3c7f83b746, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-13 00:55:36,079 INFO [Listener at localhost/37869] throttle.PressureAwareThroughputController(145): 6b485aea7037b0c8ec399d8612e4ff1d#family1#compaction#109 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:55:36,136 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/40cc4bcf9b0e42b18fdb316683d81e79, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/b23fab6f21d54fc79bf2d85aaa21d98d, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/ba1edce2256940b0ab26d5d2235ebf4e, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/ad7c82764fe141a49defe528f6bd7f78, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/bd0d461eb72541228181e06fef48cf9c] to archive 2023-02-13 00:55:36,136 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:36,138 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/40cc4bcf9b0e42b18fdb316683d81e79 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/40cc4bcf9b0e42b18fdb316683d81e79 2023-02-13 00:55:36,139 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/b23fab6f21d54fc79bf2d85aaa21d98d to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/b23fab6f21d54fc79bf2d85aaa21d98d 2023-02-13 00:55:36,140 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/ba1edce2256940b0ab26d5d2235ebf4e to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/ba1edce2256940b0ab26d5d2235ebf4e 2023-02-13 00:55:36,141 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/ad7c82764fe141a49defe528f6bd7f78 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/ad7c82764fe141a49defe528f6bd7f78 2023-02-13 00:55:36,142 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/bd0d461eb72541228181e06fef48cf9c to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/bd0d461eb72541228181e06fef48cf9c 2023-02-13 00:55:36,488 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023021386faa3544c7f4034b615263bcce88e79_del 2023-02-13 00:55:36,489 INFO [Listener at localhost/37869] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021386faa3544c7f4034b615263bcce88e79_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021386faa3544c7f4034b615263bcce88e79_del 2023-02-13 00:55:37,304 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dcbded860d4c477888e3c944767dd81f 2023-02-13 00:55:37,304 DEBUG [Listener at localhost/37869] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp/family1/dcbded860d4c477888e3c944767dd81f as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/dcbded860d4c477888e3c944767dd81f 2023-02-13 00:55:37,308 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dcbded860d4c477888e3c944767dd81f 2023-02-13 00:55:37,308 INFO [Listener at localhost/37869] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6b485aea7037b0c8ec399d8612e4ff1d/family1 of 6b485aea7037b0c8ec399d8612e4ff1d into dcbded860d4c477888e3c944767dd81f(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 00:55:37,308 DEBUG [Listener at localhost/37869] regionserver.HRegion(2288): Compaction status journal for 6b485aea7037b0c8ec399d8612e4ff1d: 2023-02-13 00:55:37,309 DEBUG [Listener at localhost/37869] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 00:55:37,309 DEBUG [Listener at localhost/37869] regionserver.HStore(1912): 6b485aea7037b0c8ec399d8612e4ff1d/family2 is initiating major compaction (all files) 2023-02-13 00:55:37,309 INFO [Listener at localhost/37869] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 00:55:37,309 INFO [Listener at localhost/37869] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:55:37,309 INFO [Listener at localhost/37869] regionserver.HRegion(2258): Starting compaction of 6b485aea7037b0c8ec399d8612e4ff1d/family2 in testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:37,310 INFO [Listener at localhost/37869] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/c012deef21c64190a31b6c1d6cd799b4, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/d56089120eee43ac9a8f4bee550a86d0, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/9124faf925464c62955abcbcf2b28843, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/d095b833edb04e21a0f6de413b1e43f9, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/671b924f27c74cc7918fc49264df3b33] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp, totalSize=25.4 K 2023-02-13 00:55:37,310 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting c012deef21c64190a31b6c1d6cd799b4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676249720884 2023-02-13 00:55:37,310 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting d56089120eee43ac9a8f4bee550a86d0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676249722686 2023-02-13 00:55:37,311 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 9124faf925464c62955abcbcf2b28843, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676249724465 2023-02-13 00:55:37,311 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting d095b833edb04e21a0f6de413b1e43f9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676249726252 2023-02-13 00:55:37,311 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 671b924f27c74cc7918fc49264df3b33, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-13 00:55:37,321 INFO [Listener at localhost/37869] throttle.PressureAwareThroughputController(145): 6b485aea7037b0c8ec399d8612e4ff1d#family2#compaction#110 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:55:37,337 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/5698639ce41540af9d712c378fe62e38, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/0ed061c653df44ce9f169ddc4b5dfa54, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/d0d7911c42c84855967dab4d508097d9, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/85892d97287240ffb900a29077361472, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/05c6a7b6fdaa414ab8db4a3c7f83b746] to archive 2023-02-13 00:55:37,337 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:37,339 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/5698639ce41540af9d712c378fe62e38 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/5698639ce41540af9d712c378fe62e38 2023-02-13 00:55:37,340 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/0ed061c653df44ce9f169ddc4b5dfa54 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/0ed061c653df44ce9f169ddc4b5dfa54 2023-02-13 00:55:37,341 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/d0d7911c42c84855967dab4d508097d9 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/d0d7911c42c84855967dab4d508097d9 2023-02-13 00:55:37,341 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/85892d97287240ffb900a29077361472 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/85892d97287240ffb900a29077361472 2023-02-13 00:55:37,342 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/05c6a7b6fdaa414ab8db4a3c7f83b746 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/05c6a7b6fdaa414ab8db4a3c7f83b746 2023-02-13 00:55:37,728 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230213eaee0f2685b844a184b87bf8faf7a0e9_del 2023-02-13 00:55:37,729 INFO [Listener at localhost/37869] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213eaee0f2685b844a184b87bf8faf7a0e9_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213eaee0f2685b844a184b87bf8faf7a0e9_del 2023-02-13 00:55:38,542 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 400ea6b3334b476294cee24f0da79ff9 2023-02-13 00:55:38,543 DEBUG [Listener at localhost/37869] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp/family2/400ea6b3334b476294cee24f0da79ff9 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/400ea6b3334b476294cee24f0da79ff9 2023-02-13 00:55:38,546 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 400ea6b3334b476294cee24f0da79ff9 2023-02-13 00:55:38,547 INFO [Listener at localhost/37869] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6b485aea7037b0c8ec399d8612e4ff1d/family2 of 6b485aea7037b0c8ec399d8612e4ff1d into 400ea6b3334b476294cee24f0da79ff9(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 00:55:38,547 DEBUG [Listener at localhost/37869] regionserver.HRegion(2288): Compaction status journal for 6b485aea7037b0c8ec399d8612e4ff1d: 2023-02-13 00:55:38,547 DEBUG [Listener at localhost/37869] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 00:55:38,547 DEBUG [Listener at localhost/37869] regionserver.HStore(1912): 89b71b72a624c785439c7640cd6fb013/family1 is initiating major compaction (all files) 2023-02-13 00:55:38,547 INFO [Listener at localhost/37869] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 00:55:38,547 INFO [Listener at localhost/37869] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:55:38,548 INFO [Listener at localhost/37869] regionserver.HRegion(2258): Starting compaction of 89b71b72a624c785439c7640cd6fb013/family1 in testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:38,548 INFO [Listener at localhost/37869] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/b764582729ff4ae3a7b761976a0a3c41, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/42284400d5f9487b972af1c61186b106, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/fd70930d268d4ba6afdcdacce99ab021, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/04385aa81e7440e7a4d326422c60873f, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/fb94a170d3444262b51b05c3598b0642] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp, totalSize=26.7 K 2023-02-13 00:55:38,549 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting b764582729ff4ae3a7b761976a0a3c41, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676249720883 2023-02-13 00:55:38,549 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 42284400d5f9487b972af1c61186b106, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676249722686 2023-02-13 00:55:38,549 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting fd70930d268d4ba6afdcdacce99ab021, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676249724465 2023-02-13 00:55:38,549 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 04385aa81e7440e7a4d326422c60873f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676249726252 2023-02-13 00:55:38,550 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting fb94a170d3444262b51b05c3598b0642, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-13 00:55:38,559 INFO [Listener at localhost/37869] throttle.PressureAwareThroughputController(145): 89b71b72a624c785439c7640cd6fb013#family1#compaction#111 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:55:38,636 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/c012deef21c64190a31b6c1d6cd799b4, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/d56089120eee43ac9a8f4bee550a86d0, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/9124faf925464c62955abcbcf2b28843, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/d095b833edb04e21a0f6de413b1e43f9, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/671b924f27c74cc7918fc49264df3b33] to archive 2023-02-13 00:55:38,637 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:38,638 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/c012deef21c64190a31b6c1d6cd799b4 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/c012deef21c64190a31b6c1d6cd799b4 2023-02-13 00:55:38,639 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/d56089120eee43ac9a8f4bee550a86d0 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/d56089120eee43ac9a8f4bee550a86d0 2023-02-13 00:55:38,640 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/9124faf925464c62955abcbcf2b28843 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/9124faf925464c62955abcbcf2b28843 2023-02-13 00:55:38,641 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/d095b833edb04e21a0f6de413b1e43f9 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/d095b833edb04e21a0f6de413b1e43f9 2023-02-13 00:55:38,642 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/671b924f27c74cc7918fc49264df3b33 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/671b924f27c74cc7918fc49264df3b33 2023-02-13 00:55:38,967 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023021338b5d5a4488548e885eb22a8c59bec81_del 2023-02-13 00:55:38,967 INFO [Listener at localhost/37869] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021338b5d5a4488548e885eb22a8c59bec81_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021338b5d5a4488548e885eb22a8c59bec81_del 2023-02-13 00:55:39,780 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 83ee4d116ee54b588337c72d361fb063 2023-02-13 00:55:39,781 DEBUG [Listener at localhost/37869] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp/family1/83ee4d116ee54b588337c72d361fb063 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/83ee4d116ee54b588337c72d361fb063 2023-02-13 00:55:39,785 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 83ee4d116ee54b588337c72d361fb063 2023-02-13 00:55:39,786 INFO [Listener at localhost/37869] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 89b71b72a624c785439c7640cd6fb013/family1 of 89b71b72a624c785439c7640cd6fb013 into 83ee4d116ee54b588337c72d361fb063(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 00:55:39,786 DEBUG [Listener at localhost/37869] regionserver.HRegion(2288): Compaction status journal for 89b71b72a624c785439c7640cd6fb013: 2023-02-13 00:55:39,786 DEBUG [Listener at localhost/37869] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 00:55:39,786 DEBUG [Listener at localhost/37869] regionserver.HStore(1912): 89b71b72a624c785439c7640cd6fb013/family2 is initiating major compaction (all files) 2023-02-13 00:55:39,786 INFO [Listener at localhost/37869] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 00:55:39,787 INFO [Listener at localhost/37869] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:55:39,787 INFO [Listener at localhost/37869] regionserver.HRegion(2258): Starting compaction of 89b71b72a624c785439c7640cd6fb013/family2 in testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:39,787 INFO [Listener at localhost/37869] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/a0f7d35e413a49d6bf0acc6f8d6693b6, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/8b40a4cff14a42e1b4f7b39a7371f146, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/5aa6bed313e74ae0b57bb6aba6db12aa, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/b136ea849d0e4e6a8cbe2bb45e6ac4b8, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/a4164f603b5c467181f95a60bd292b8b] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp, totalSize=25.4 K 2023-02-13 00:55:39,787 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting a0f7d35e413a49d6bf0acc6f8d6693b6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676249720883 2023-02-13 00:55:39,788 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 8b40a4cff14a42e1b4f7b39a7371f146, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676249722686 2023-02-13 00:55:39,788 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 5aa6bed313e74ae0b57bb6aba6db12aa, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676249724465 2023-02-13 00:55:39,788 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting b136ea849d0e4e6a8cbe2bb45e6ac4b8, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676249726252 2023-02-13 00:55:39,789 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting a4164f603b5c467181f95a60bd292b8b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-13 00:55:39,799 INFO [Listener at localhost/37869] throttle.PressureAwareThroughputController(145): 89b71b72a624c785439c7640cd6fb013#family2#compaction#112 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:55:39,836 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/b764582729ff4ae3a7b761976a0a3c41, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/42284400d5f9487b972af1c61186b106, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/fd70930d268d4ba6afdcdacce99ab021, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/04385aa81e7440e7a4d326422c60873f, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/fb94a170d3444262b51b05c3598b0642] to archive 2023-02-13 00:55:39,837 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:39,839 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/b764582729ff4ae3a7b761976a0a3c41 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/b764582729ff4ae3a7b761976a0a3c41 2023-02-13 00:55:39,840 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/42284400d5f9487b972af1c61186b106 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/42284400d5f9487b972af1c61186b106 2023-02-13 00:55:39,841 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/fd70930d268d4ba6afdcdacce99ab021 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/fd70930d268d4ba6afdcdacce99ab021 2023-02-13 00:55:39,842 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/04385aa81e7440e7a4d326422c60873f to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/04385aa81e7440e7a4d326422c60873f 2023-02-13 00:55:39,843 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/fb94a170d3444262b51b05c3598b0642 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/fb94a170d3444262b51b05c3598b0642 2023-02-13 00:55:40,207 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302134f254ae4f6754a24a83eeb01a5e5e7d1_del 2023-02-13 00:55:40,207 INFO [Listener at localhost/37869] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302134f254ae4f6754a24a83eeb01a5e5e7d1_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302134f254ae4f6754a24a83eeb01a5e5e7d1_del 2023-02-13 00:55:41,022 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fa26ce03ae034f1e84d98fe6b148ca16 2023-02-13 00:55:41,022 DEBUG [Listener at localhost/37869] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp/family2/fa26ce03ae034f1e84d98fe6b148ca16 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/fa26ce03ae034f1e84d98fe6b148ca16 2023-02-13 00:55:41,026 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fa26ce03ae034f1e84d98fe6b148ca16 2023-02-13 00:55:41,027 INFO [Listener at localhost/37869] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 89b71b72a624c785439c7640cd6fb013/family2 of 89b71b72a624c785439c7640cd6fb013 into fa26ce03ae034f1e84d98fe6b148ca16(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 00:55:41,027 DEBUG [Listener at localhost/37869] regionserver.HRegion(2288): Compaction status journal for 89b71b72a624c785439c7640cd6fb013: 2023-02-13 00:55:41,032 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 00:55:41,035 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 00:55:41,036 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/a0f7d35e413a49d6bf0acc6f8d6693b6, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/8b40a4cff14a42e1b4f7b39a7371f146, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/5aa6bed313e74ae0b57bb6aba6db12aa, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/b136ea849d0e4e6a8cbe2bb45e6ac4b8, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/a4164f603b5c467181f95a60bd292b8b] to archive 2023-02-13 00:55:41,037 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:41,039 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/a0f7d35e413a49d6bf0acc6f8d6693b6 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/a0f7d35e413a49d6bf0acc6f8d6693b6 2023-02-13 00:55:41,040 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/8b40a4cff14a42e1b4f7b39a7371f146 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/8b40a4cff14a42e1b4f7b39a7371f146 2023-02-13 00:55:41,040 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 00:55:41,041 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 00:55:41,042 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/5aa6bed313e74ae0b57bb6aba6db12aa to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/5aa6bed313e74ae0b57bb6aba6db12aa 2023-02-13 00:55:41,043 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/b136ea849d0e4e6a8cbe2bb45e6ac4b8 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/b136ea849d0e4e6a8cbe2bb45e6ac4b8 2023-02-13 00:55:41,044 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/a4164f603b5c467181f95a60bd292b8b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/a4164f603b5c467181f95a60bd292b8b 2023-02-13 00:55:41,047 INFO [Listener at localhost/37869] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-13 00:55:41,082 INFO [Listener at localhost/37869] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-02-13 00:55:41,085 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023021338b5d5a4488548e885eb22a8c59bec81_del 2023-02-13 00:55:41,088 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023021386faa3544c7f4034b615263bcce88e79_del 2023-02-13 00:55:41,091 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023021309328d5dd03c45fa9f52ce41e3e16351_del 2023-02-13 00:55:41,091 INFO [Listener at localhost/37869] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-02-13 00:55:41,091 DEBUG [Listener at localhost/37869] zookeeper.ReadOnlyZKClient(139): Connect 0x314f3d9c to 127.0.0.1:54478 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:55:41,096 DEBUG [Listener at localhost/37869] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2a8e9e10, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:55:41,096 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 00:55:41,096 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 00:55:41,096 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 00:55:41,550 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302139e971c2ffc394f7fb8957aa9b9cca598 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302139e971c2ffc394f7fb8957aa9b9cca598 2023-02-13 00:55:41,550 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302133b38d90ede5d4a34a87f04b7a7f97204 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302133b38d90ede5d4a34a87f04b7a7f97204 2023-02-13 00:55:41,551 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 00:55:41,551 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 00:55:41,567 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:55:41,570 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42068, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:55:41,576 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 00:55:41,578 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49312, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 00:55:41,588 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213/family1/58e4aab4c74b40bc8ded712998835208 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-02-13 00:55:41,588 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213/family1/f5818de8516241d5a5d30955a766d60d first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-02-13 00:55:41,589 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213/family1/f5818de8516241d5a5d30955a766d60d}] 2023-02-13 00:55:41,589 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213/family1/58e4aab4c74b40bc8ded712998835208}] 2023-02-13 00:55:41,594 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213/family1/f5818de8516241d5a5d30955a766d60d for inclusion in 6b485aea7037b0c8ec399d8612e4ff1d/family1 2023-02-13 00:55:41,594 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213/family1/58e4aab4c74b40bc8ded712998835208 for inclusion in 63f9bed78ea4a19108ec7ecea54182e3/family1 2023-02-13 00:55:41,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-02-13 00:55:41,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-13 00:55:41,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-02-13 00:55:41,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-13 00:55:41,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for 63f9bed78ea4a19108ec7ecea54182e3: 2023-02-13 00:55:41,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for 6b485aea7037b0c8ec399d8612e4ff1d: 2023-02-13 00:55:41,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213/family1/58e4aab4c74b40bc8ded712998835208 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__ilffs4pph40fsgndgi696hk37243pps94mheo7ksfbkrr120l5i6j1g2a4fve42o/family1/58e4aab4c74b40bc8ded712998835208 2023-02-13 00:55:41,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213/family1/f5818de8516241d5a5d30955a766d60d to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__mmg7pjc73isde72ru2ucvbpfhhui76si3bdutamu77j2ng5hh1jd585r3uirnivd/family1/f5818de8516241d5a5d30955a766d60d 2023-02-13 00:55:41,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__ilffs4pph40fsgndgi696hk37243pps94mheo7ksfbkrr120l5i6j1g2a4fve42o/family1/58e4aab4c74b40bc8ded712998835208 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/bc70e5a3863b47d0920da21823aeb1bf_SeqId_29_ 2023-02-13 00:55:41,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__mmg7pjc73isde72ru2ucvbpfhhui76si3bdutamu77j2ng5hh1jd585r3uirnivd/family1/f5818de8516241d5a5d30955a766d60d as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/36e3c77524c045f1b5bb0e77ef72df16_SeqId_29_ 2023-02-13 00:55:41,599 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__ilffs4pph40fsgndgi696hk37243pps94mheo7ksfbkrr120l5i6j1g2a4fve42o/family1/58e4aab4c74b40bc8ded712998835208 into 63f9bed78ea4a19108ec7ecea54182e3/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/bc70e5a3863b47d0920da21823aeb1bf_SeqId_29_ - updating store file list. 2023-02-13 00:55:41,599 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__mmg7pjc73isde72ru2ucvbpfhhui76si3bdutamu77j2ng5hh1jd585r3uirnivd/family1/f5818de8516241d5a5d30955a766d60d into 6b485aea7037b0c8ec399d8612e4ff1d/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/36e3c77524c045f1b5bb0e77ef72df16_SeqId_29_ - updating store file list. 2023-02-13 00:55:41,602 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/36e3c77524c045f1b5bb0e77ef72df16_SeqId_29_ into 6b485aea7037b0c8ec399d8612e4ff1d/family1 2023-02-13 00:55:41,602 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__mmg7pjc73isde72ru2ucvbpfhhui76si3bdutamu77j2ng5hh1jd585r3uirnivd/family1/f5818de8516241d5a5d30955a766d60d into 6b485aea7037b0c8ec399d8612e4ff1d/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/36e3c77524c045f1b5bb0e77ef72df16_SeqId_29_) 2023-02-13 00:55:41,602 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/bc70e5a3863b47d0920da21823aeb1bf_SeqId_29_ into 63f9bed78ea4a19108ec7ecea54182e3/family1 2023-02-13 00:55:41,602 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__ilffs4pph40fsgndgi696hk37243pps94mheo7ksfbkrr120l5i6j1g2a4fve42o/family1/58e4aab4c74b40bc8ded712998835208 into 63f9bed78ea4a19108ec7ecea54182e3/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/bc70e5a3863b47d0920da21823aeb1bf_SeqId_29_) 2023-02-13 00:55:41,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__mmg7pjc73isde72ru2ucvbpfhhui76si3bdutamu77j2ng5hh1jd585r3uirnivd/family1/f5818de8516241d5a5d30955a766d60d 2023-02-13 00:55:41,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__ilffs4pph40fsgndgi696hk37243pps94mheo7ksfbkrr120l5i6j1g2a4fve42o/family1/58e4aab4c74b40bc8ded712998835208 2023-02-13 00:55:41,604 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 00:55:41,604 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 00:55:41,605 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:41,605 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:41,606 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302136a074e5589c4444cbea4ec51c8c112f7 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302136a074e5589c4444cbea4ec51c8c112f7 2023-02-13 00:55:41,606 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302131dc8db5fc31c4457bc825856a90efcb6 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302131dc8db5fc31c4457bc825856a90efcb6 2023-02-13 00:55:41,607 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213c7c589f59d5a46e69f929bfe1462e530 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213c7c589f59d5a46e69f929bfe1462e530 2023-02-13 00:55:41,607 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302132917a3ddef6a4d7e847180bde371f97b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302132917a3ddef6a4d7e847180bde371f97b 2023-02-13 00:55:41,608 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302133736609080864ed1af2fda92d4e1087b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302133736609080864ed1af2fda92d4e1087b 2023-02-13 00:55:41,608 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213e74bd61456ae4f18ae9a602ef3d7ea75 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213e74bd61456ae4f18ae9a602ef3d7ea75 2023-02-13 00:55:41,609 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302139ab092340bd14dbc8b723519529420ae to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302139ab092340bd14dbc8b723519529420ae 2023-02-13 00:55:41,609 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-13 00:55:41,609 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213f09c030f2155421aa571ae8f4246407d to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213f09c030f2155421aa571ae8f4246407d 2023-02-13 00:55:41,609 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-13 00:55:41,609 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213' 2023-02-13 00:55:41,609 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213' 2023-02-13 00:55:41,610 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 00:55:41,610 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 00:55:41,949 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302134ee95288cade446c932a4d686584eeed to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302134ee95288cade446c932a4d686584eeed 2023-02-13 00:55:41,950 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 00:55:41,974 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213/family1/13317abefd0c46168e80082674aeeb9b first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-02-13 00:55:41,975 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213/family1/13317abefd0c46168e80082674aeeb9b}] 2023-02-13 00:55:41,977 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213/family1/13317abefd0c46168e80082674aeeb9b for inclusion in 89b71b72a624c785439c7640cd6fb013/family1 2023-02-13 00:55:41,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-02-13 00:55:41,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(777): Region bounds: first= last=1 2023-02-13 00:55:41,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for 89b71b72a624c785439c7640cd6fb013: 2023-02-13 00:55:41,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213/family1/13317abefd0c46168e80082674aeeb9b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__9lem5jd25dg69nuk4kdos9k7ae4gmv051momqbkl9h4mr937k17b6ft5pf9mdfnc/family1/13317abefd0c46168e80082674aeeb9b 2023-02-13 00:55:41,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__9lem5jd25dg69nuk4kdos9k7ae4gmv051momqbkl9h4mr937k17b6ft5pf9mdfnc/family1/13317abefd0c46168e80082674aeeb9b as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/a159d022ebbb421f861260b2f0731b4d_SeqId_29_ 2023-02-13 00:55:41,985 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__9lem5jd25dg69nuk4kdos9k7ae4gmv051momqbkl9h4mr937k17b6ft5pf9mdfnc/family1/13317abefd0c46168e80082674aeeb9b into 89b71b72a624c785439c7640cd6fb013/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/a159d022ebbb421f861260b2f0731b4d_SeqId_29_ - updating store file list. 2023-02-13 00:55:41,988 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/a159d022ebbb421f861260b2f0731b4d_SeqId_29_ into 89b71b72a624c785439c7640cd6fb013/family1 2023-02-13 00:55:41,989 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__9lem5jd25dg69nuk4kdos9k7ae4gmv051momqbkl9h4mr937k17b6ft5pf9mdfnc/family1/13317abefd0c46168e80082674aeeb9b into 89b71b72a624c785439c7640cd6fb013/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/a159d022ebbb421f861260b2f0731b4d_SeqId_29_) 2023-02-13 00:55:41,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__9lem5jd25dg69nuk4kdos9k7ae4gmv051momqbkl9h4mr937k17b6ft5pf9mdfnc/family1/13317abefd0c46168e80082674aeeb9b 2023-02-13 00:55:41,991 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 00:55:41,992 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:41,993 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021308341f081e314a2dae1e3f07ac8a389b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021308341f081e314a2dae1e3f07ac8a389b 2023-02-13 00:55:41,995 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021322eec433a1514cdaaa42c42305d30a12 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021322eec433a1514cdaaa42c42305d30a12 2023-02-13 00:55:41,996 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213348ca6e4353c43259c1482c61f82b4a5 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213348ca6e4353c43259c1482c61f82b4a5 2023-02-13 00:55:41,997 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302138f21581a0c4c4b58a0ad086df3e0ae67 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302138f21581a0c4c4b58a0ad086df3e0ae67 2023-02-13 00:55:41,997 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-13 00:55:41,997 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213' 2023-02-13 00:55:41,997 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 00:55:41,997 INFO [Listener at localhost/37869] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-02-13 00:55:41,998 INFO [Listener at localhost/37869] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 00:55:41,998 INFO [Listener at localhost/37869] compactions.PartitionedMobCompactor(367): [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021338b5d5a4488548e885eb22a8c59bec81_del] 2023-02-13 00:55:41,998 DEBUG [Listener at localhost/37869] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:42,000 DEBUG [Listener at localhost/37869] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021338b5d5a4488548e885eb22a8c59bec81_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021338b5d5a4488548e885eb22a8c59bec81_del 2023-02-13 00:55:42,000 INFO [Listener at localhost/37869] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-13 00:55:42,000 INFO [Listener at localhost/37869] compactions.PartitionedMobCompactor(367): [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021386faa3544c7f4034b615263bcce88e79_del] 2023-02-13 00:55:42,001 DEBUG [Listener at localhost/37869] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:42,002 DEBUG [Listener at localhost/37869] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021386faa3544c7f4034b615263bcce88e79_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021386faa3544c7f4034b615263bcce88e79_del 2023-02-13 00:55:42,003 INFO [Listener at localhost/37869] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-13 00:55:42,003 INFO [Listener at localhost/37869] compactions.PartitionedMobCompactor(367): [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021309328d5dd03c45fa9f52ce41e3e16351_del] 2023-02-13 00:55:42,003 DEBUG [Listener at localhost/37869] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:42,004 DEBUG [Listener at localhost/37869] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021309328d5dd03c45fa9f52ce41e3e16351_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021309328d5dd03c45fa9f52ce41e3e16351_del 2023-02-13 00:55:42,005 INFO [Listener at localhost/37869] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-13 00:55:42,007 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 00:55:42,010 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 00:55:42,014 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 00:55:42,015 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 00:55:42,021 INFO [Listener at localhost/37869] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-02-13 00:55:42,022 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testCompactionWithHFileLink 2023-02-13 00:55:42,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-02-13 00:55:42,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 00:55:42,025 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249742025"}]},"ts":"1676249742025"} 2023-02-13 00:55:42,027 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-02-13 00:55:42,029 INFO [PEWorker-3] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-02-13 00:55:42,030 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=89b71b72a624c785439c7640cd6fb013, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6b485aea7037b0c8ec399d8612e4ff1d, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=63f9bed78ea4a19108ec7ecea54182e3, UNASSIGN}] 2023-02-13 00:55:42,031 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=63f9bed78ea4a19108ec7ecea54182e3, UNASSIGN 2023-02-13 00:55:42,031 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6b485aea7037b0c8ec399d8612e4ff1d, UNASSIGN 2023-02-13 00:55:42,031 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=89b71b72a624c785439c7640cd6fb013, UNASSIGN 2023-02-13 00:55:42,032 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=63f9bed78ea4a19108ec7ecea54182e3, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:42,032 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=89b71b72a624c785439c7640cd6fb013, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:42,032 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676249742032"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249742032"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249742032"}]},"ts":"1676249742032"} 2023-02-13 00:55:42,032 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676249742032"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249742032"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249742032"}]},"ts":"1676249742032"} 2023-02-13 00:55:42,032 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=6b485aea7037b0c8ec399d8612e4ff1d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:42,032 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676249742032"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249742032"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249742032"}]},"ts":"1676249742032"} 2023-02-13 00:55:42,033 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=91, state=RUNNABLE; CloseRegionProcedure 63f9bed78ea4a19108ec7ecea54182e3, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:55:42,034 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=89, state=RUNNABLE; CloseRegionProcedure 89b71b72a624c785439c7640cd6fb013, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:55:42,034 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=90, state=RUNNABLE; CloseRegionProcedure 6b485aea7037b0c8ec399d8612e4ff1d, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:55:42,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 00:55:42,184 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 63f9bed78ea4a19108ec7ecea54182e3 2023-02-13 00:55:42,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 63f9bed78ea4a19108ec7ecea54182e3, disabling compactions & flushes 2023-02-13 00:55:42,185 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:42,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:42,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. after waiting 0 ms 2023-02-13 00:55:42,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:42,189 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-13 00:55:42,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:42,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 63f9bed78ea4a19108ec7ecea54182e3: Waiting for close lock at 1676249742184Running coprocessor pre-close hooks at 1676249742184Disabling compacts and flushes for region at 1676249742184Disabling writes for close at 1676249742185 (+1 ms)Writing region close event to WAL at 1676249742186 (+1 ms)Running coprocessor post-close hooks at 1676249742190 (+4 ms)Closed at 1676249742190 2023-02-13 00:55:42,192 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 63f9bed78ea4a19108ec7ecea54182e3 2023-02-13 00:55:42,192 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 6b485aea7037b0c8ec399d8612e4ff1d 2023-02-13 00:55:42,192 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 6b485aea7037b0c8ec399d8612e4ff1d, disabling compactions & flushes 2023-02-13 00:55:42,192 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:42,192 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:42,192 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. after waiting 0 ms 2023-02-13 00:55:42,192 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:42,192 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=63f9bed78ea4a19108ec7ecea54182e3, regionState=CLOSED 2023-02-13 00:55:42,192 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676249742192"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249742192"}]},"ts":"1676249742192"} 2023-02-13 00:55:42,195 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=91 2023-02-13 00:55:42,195 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=91, state=SUCCESS; CloseRegionProcedure 63f9bed78ea4a19108ec7ecea54182e3, server=jenkins-hbase4.apache.org,36605,1676249622318 in 161 msec 2023-02-13 00:55:42,196 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-13 00:55:42,196 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=63f9bed78ea4a19108ec7ecea54182e3, UNASSIGN in 165 msec 2023-02-13 00:55:42,197 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:42,197 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 6b485aea7037b0c8ec399d8612e4ff1d: Waiting for close lock at 1676249742192Running coprocessor pre-close hooks at 1676249742192Disabling compacts and flushes for region at 1676249742192Disabling writes for close at 1676249742192Writing region close event to WAL at 1676249742193 (+1 ms)Running coprocessor post-close hooks at 1676249742196 (+3 ms)Closed at 1676249742197 (+1 ms) 2023-02-13 00:55:42,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 6b485aea7037b0c8ec399d8612e4ff1d 2023-02-13 00:55:42,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 89b71b72a624c785439c7640cd6fb013 2023-02-13 00:55:42,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 89b71b72a624c785439c7640cd6fb013, disabling compactions & flushes 2023-02-13 00:55:42,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:42,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:42,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. after waiting 0 ms 2023-02-13 00:55:42,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:42,198 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=6b485aea7037b0c8ec399d8612e4ff1d, regionState=CLOSED 2023-02-13 00:55:42,198 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676249742198"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249742198"}]},"ts":"1676249742198"} 2023-02-13 00:55:42,201 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=90 2023-02-13 00:55:42,201 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=90, state=SUCCESS; CloseRegionProcedure 6b485aea7037b0c8ec399d8612e4ff1d, server=jenkins-hbase4.apache.org,36605,1676249622318 in 165 msec 2023-02-13 00:55:42,201 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6b485aea7037b0c8ec399d8612e4ff1d, UNASSIGN in 171 msec 2023-02-13 00:55:42,202 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-13 00:55:42,203 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:42,203 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 89b71b72a624c785439c7640cd6fb013: Waiting for close lock at 1676249742198Running coprocessor pre-close hooks at 1676249742198Disabling compacts and flushes for region at 1676249742198Disabling writes for close at 1676249742198Writing region close event to WAL at 1676249742199 (+1 ms)Running coprocessor post-close hooks at 1676249742202 (+3 ms)Closed at 1676249742203 (+1 ms) 2023-02-13 00:55:42,204 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 89b71b72a624c785439c7640cd6fb013 2023-02-13 00:55:42,204 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=89b71b72a624c785439c7640cd6fb013, regionState=CLOSED 2023-02-13 00:55:42,204 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676249742204"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249742204"}]},"ts":"1676249742204"} 2023-02-13 00:55:42,206 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=89 2023-02-13 00:55:42,206 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=89, state=SUCCESS; CloseRegionProcedure 89b71b72a624c785439c7640cd6fb013, server=jenkins-hbase4.apache.org,36605,1676249622318 in 171 msec 2023-02-13 00:55:42,207 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-13 00:55:42,207 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=89b71b72a624c785439c7640cd6fb013, UNASSIGN in 176 msec 2023-02-13 00:55:42,208 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249742208"}]},"ts":"1676249742208"} 2023-02-13 00:55:42,209 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-02-13 00:55:42,210 INFO [PEWorker-4] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-02-13 00:55:42,212 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 189 msec 2023-02-13 00:55:42,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 00:55:42,328 INFO [Listener at localhost/37869] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-02-13 00:55:42,338 INFO [Listener at localhost/37869] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1676249728028-1676249742337 2023-02-13 00:55:42,339 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=hbase-failsafe-snaptb-1676249728028-1676249742337 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-13 00:55:42,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1676249742339 (current time:1676249742339). 2023-02-13 00:55:42,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-02-13 00:55:42,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] zookeeper.ReadOnlyZKClient(139): Connect 0x5b8cbca1 to 127.0.0.1:54478 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:55:42,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@76880b3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:55:42,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:55:42,347 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42076, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:55:42,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5b8cbca1 to 127.0.0.1:54478 2023-02-13 00:55:42,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:55:42,348 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-02-13 00:55:42,348 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] snapshot.SnapshotManager(659): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1676249728028-1676249742337 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-13 00:55:42,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-13 00:55:42,351 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-13 00:55:42,352 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1676250342352 2023-02-13 00:55:42,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] snapshot.SnapshotManager(664): Started snapshot: { ss=hbase-failsafe-snaptb-1676249728028-1676249742337 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-13 00:55:42,352 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1676249728028-1676249742337 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-02-13 00:55:42,352 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1676249728028-1676249742337 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-02-13 00:55:42,352 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-02-13 00:55:42,452 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 00:55:42,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1676249728028-1676249742337 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 00:55:42,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1676249728028-1676249742337 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 00:55:42,453 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-02-13 00:55:42,654 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 00:55:42,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1676249728028-1676249742337 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 00:55:42,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1676249728028-1676249742337 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 00:55:42,655 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-02-13 00:55:42,759 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1676249728028-1676249742337 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-02-13 00:55:42,761 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-13 00:55:42,761 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-13 00:55:42,761 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-13 00:55:42,761 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-13 00:55:42,761 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-13 00:55:42,761 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-13 00:55:42,761 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-13 00:55:42,761 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-13 00:55:42,765 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/3a504ca96acf4c6a9a3dbc93c6381047, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/bc70e5a3863b47d0920da21823aeb1bf_SeqId_29_] hfiles 2023-02-13 00:55:42,765 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/3a504ca96acf4c6a9a3dbc93c6381047 2023-02-13 00:55:42,765 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/bc70e5a3863b47d0920da21823aeb1bf_SeqId_29_ 2023-02-13 00:55:42,766 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/36e3c77524c045f1b5bb0e77ef72df16_SeqId_29_, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/dcbded860d4c477888e3c944767dd81f] hfiles 2023-02-13 00:55:42,766 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/36e3c77524c045f1b5bb0e77ef72df16_SeqId_29_ 2023-02-13 00:55:42,766 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302139e971c2ffc394f7fb8957aa9b9cca598, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302133b38d90ede5d4a34a87f04b7a7f97204, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302134ee95288cade446c932a4d686584eeed] hfiles 2023-02-13 00:55:42,766 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/dcbded860d4c477888e3c944767dd81f 2023-02-13 00:55:42,766 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302139e971c2ffc394f7fb8957aa9b9cca598 2023-02-13 00:55:42,766 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302133b38d90ede5d4a34a87f04b7a7f97204 2023-02-13 00:55:42,766 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/83ee4d116ee54b588337c72d361fb063, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/a159d022ebbb421f861260b2f0731b4d_SeqId_29_] hfiles 2023-02-13 00:55:42,766 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302134ee95288cade446c932a4d686584eeed 2023-02-13 00:55:42,766 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/83ee4d116ee54b588337c72d361fb063 2023-02-13 00:55:42,766 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/a159d022ebbb421f861260b2f0731b4d_SeqId_29_ 2023-02-13 00:55:42,766 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/01efca4802b34dc8baf2986ca31e146b] hfiles 2023-02-13 00:55:42,766 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/01efca4802b34dc8baf2986ca31e146b 2023-02-13 00:55:42,767 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/400ea6b3334b476294cee24f0da79ff9] hfiles 2023-02-13 00:55:42,767 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/400ea6b3334b476294cee24f0da79ff9 2023-02-13 00:55:42,767 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/fa26ce03ae034f1e84d98fe6b148ca16] hfiles 2023-02-13 00:55:42,767 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/fa26ce03ae034f1e84d98fe6b148ca16 2023-02-13 00:55:42,775 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302131e36bef117a741018eeabfe57867ea24, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023021321d6790068c3494193aa9a5eb201e991, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023021324cbfacae9a5448894c3419f66a851e0, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302134f08f76cbaf1432b9c2e01f3aeffa05c, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213eaee0f2685b844a184b87bf8faf7a0e9_del, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023021329cbdffeaba74431b13e6040dd8db724, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213d3021403671e42d48df383dfdac034be, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213edaa017cbda74487bf56b12844dd7b47, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213f7e2ebfd9e7446a4ad37f916104aed78, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213fb3cf86ef18546138ec770b18d54c288_del, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023021301e50cc3adca4f52a6f0a591b7de5f77, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302134f254ae4f6754a24a83eeb01a5e5e7d1_del, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302134f35cbbadd6c42a2b8fec88a5e65b1cf, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213c108e8256456461b958cb887a71298ec, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213e50db5b4ac5543ffa6bffd3c50af6216] hfiles 2023-02-13 00:55:42,775 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302131e36bef117a741018eeabfe57867ea24 2023-02-13 00:55:42,775 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023021321d6790068c3494193aa9a5eb201e991 2023-02-13 00:55:42,775 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023021324cbfacae9a5448894c3419f66a851e0 2023-02-13 00:55:42,775 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302134f08f76cbaf1432b9c2e01f3aeffa05c 2023-02-13 00:55:42,775 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213eaee0f2685b844a184b87bf8faf7a0e9_del 2023-02-13 00:55:42,775 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023021329cbdffeaba74431b13e6040dd8db724 2023-02-13 00:55:42,775 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213d3021403671e42d48df383dfdac034be 2023-02-13 00:55:42,775 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213edaa017cbda74487bf56b12844dd7b47 2023-02-13 00:55:42,775 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213f7e2ebfd9e7446a4ad37f916104aed78 2023-02-13 00:55:42,775 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213fb3cf86ef18546138ec770b18d54c288_del 2023-02-13 00:55:42,775 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023021301e50cc3adca4f52a6f0a591b7de5f77 2023-02-13 00:55:42,775 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302134f254ae4f6754a24a83eeb01a5e5e7d1_del 2023-02-13 00:55:42,775 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302134f35cbbadd6c42a2b8fec88a5e65b1cf 2023-02-13 00:55:42,775 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213c108e8256456461b958cb887a71298ec 2023-02-13 00:55:42,775 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213e50db5b4ac5543ffa6bffd3c50af6216 2023-02-13 00:55:42,955 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 00:55:42,956 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1676249728028-1676249742337 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 00:55:42,956 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1676249728028-1676249742337 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 00:55:42,956 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-02-13 00:55:43,174 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1676249728028-1676249742337 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-02-13 00:55:43,174 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1676249728028-1676249742337 2023-02-13 00:55:43,175 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1676249728028-1676249742337 2023-02-13 00:55:43,456 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 00:55:43,457 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1676249728028-1676249742337 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 00:55:43,457 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1676249728028-1676249742337 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 00:55:43,457 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-02-13 00:55:43,596 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1676249728028-1676249742337 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.hbase-snapshot/hbase-failsafe-snaptb-1676249728028-1676249742337 2023-02-13 00:55:43,640 DEBUG [regionserver/jenkins-hbase4:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 00:55:44,413 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1676249728028-1676249742337 of table testCompactionWithHFileLink completed 2023-02-13 00:55:44,413 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1676249728028-1676249742337 2023-02-13 00:55:44,413 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1676249728028-1676249742337 2023-02-13 00:55:44,413 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1676249728028-1676249742337 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1676249742352Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1676249728028-1676249742337 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1676249742759 (+407 ms)Consolidate snapshot: hbase-failsafe-snaptb-1676249728028-1676249742337 at 1676249743174 (+415 ms)Loading Region manifests for hbase-failsafe-snaptb-1676249728028-1676249742337 at 1676249743174Writing data manifest for hbase-failsafe-snaptb-1676249728028-1676249742337 at 1676249743177 (+3 ms)Verifying snapshot: hbase-failsafe-snaptb-1676249728028-1676249742337 at 1676249743582 (+405 ms)Snapshot hbase-failsafe-snaptb-1676249728028-1676249742337 of table testCompactionWithHFileLink completed at 1676249744413 (+831 ms) 2023-02-13 00:55:44,414 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-13 00:55:44,416 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 2.0640 sec 2023-02-13 00:55:44,457 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 00:55:44,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1676249728028-1676249742337 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 00:55:44,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] snapshot.SnapshotManager(401): Snapshot '{ ss=hbase-failsafe-snaptb-1676249728028-1676249742337 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-02-13 00:55:44,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1676249728028" table: "testCompactionWithHFileLink" creation_time: 1676249728038 type: FLUSH version: 2 ttl: 0 ) 2023-02-13 00:55:44,497 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] snapshot.SnapshotManager(880): Restore snapshot=snaptb-1676249728028 as table=testCompactionWithHFileLink 2023-02-13 00:55:44,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 00:55:44,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 00:55:44,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 00:55:44,906 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-02-13 00:55:44,907 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-02-13 00:55:44,907 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-02-13 00:55:44,908 INFO [PEWorker-5] procedure.RestoreSnapshotProcedure(374): Starting restore snapshot={ ss=snaptb-1676249728028 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-13 00:55:44,912 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1676249728028" table: "testCompactionWithHFileLink" creation_time: 1676249728038 type: FLUSH version: 2 ttl: 0 2023-02-13 00:55:44,913 DEBUG [PEWorker-5] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink 2023-02-13 00:55:44,917 DEBUG [PEWorker-5] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-02-13 00:55:44,920 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302133b38d90ede5d4a34a87f04b7a7f97204 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302133b38d90ede5d4a34a87f04b7a7f97204 2023-02-13 00:55:44,922 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302139e971c2ffc394f7fb8957aa9b9cca598 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302139e971c2ffc394f7fb8957aa9b9cca598 2023-02-13 00:55:44,923 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302134ee95288cade446c932a4d686584eeed to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302134ee95288cade446c932a4d686584eeed 2023-02-13 00:55:44,923 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202302131dc8db5fc31c4457bc825856a90efcb6 of snapshot=snaptb-1676249728028 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 00:55:44,926 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202302132917a3ddef6a4d7e847180bde371f97b of snapshot=snaptb-1676249728028 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 00:55:44,929 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202302133736609080864ed1af2fda92d4e1087b of snapshot=snaptb-1676249728028 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 00:55:44,931 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202302139ab092340bd14dbc8b723519529420ae of snapshot=snaptb-1676249728028 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 00:55:44,934 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202302136a074e5589c4444cbea4ec51c8c112f7 of snapshot=snaptb-1676249728028 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 00:55:44,936 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230213c7c589f59d5a46e69f929bfe1462e530 of snapshot=snaptb-1676249728028 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 00:55:44,939 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230213e74bd61456ae4f18ae9a602ef3d7ea75 of snapshot=snaptb-1676249728028 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 00:55:44,942 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230213f09c030f2155421aa571ae8f4246407d of snapshot=snaptb-1676249728028 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 00:55:44,945 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e2023021308341f081e314a2dae1e3f07ac8a389b of snapshot=snaptb-1676249728028 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 00:55:44,947 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e2023021322eec433a1514cdaaa42c42305d30a12 of snapshot=snaptb-1676249728028 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 00:55:44,950 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230213348ca6e4353c43259c1482c61f82b4a5 of snapshot=snaptb-1676249728028 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 00:55:44,953 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202302138f21581a0c4c4b58a0ad086df3e0ae67 of snapshot=snaptb-1676249728028 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 00:55:44,957 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302134f254ae4f6754a24a83eeb01a5e5e7d1_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302134f254ae4f6754a24a83eeb01a5e5e7d1_del 2023-02-13 00:55:44,958 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213fb3cf86ef18546138ec770b18d54c288_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213fb3cf86ef18546138ec770b18d54c288_del 2023-02-13 00:55:44,959 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213eaee0f2685b844a184b87bf8faf7a0e9_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213eaee0f2685b844a184b87bf8faf7a0e9_del 2023-02-13 00:55:44,959 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(223): region to restore: 63f9bed78ea4a19108ec7ecea54182e3 2023-02-13 00:55:44,960 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(223): region to restore: 6b485aea7037b0c8ec399d8612e4ff1d 2023-02-13 00:55:44,960 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(223): region to restore: 89b71b72a624c785439c7640cd6fb013 2023-02-13 00:55:44,963 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/bc70e5a3863b47d0920da21823aeb1bf_SeqId_29_ to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/bc70e5a3863b47d0920da21823aeb1bf_SeqId_29_ 2023-02-13 00:55:44,963 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/dcbded860d4c477888e3c944767dd81f to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/dcbded860d4c477888e3c944767dd81f 2023-02-13 00:55:44,964 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/a159d022ebbb421f861260b2f0731b4d_SeqId_29_ to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/a159d022ebbb421f861260b2f0731b4d_SeqId_29_ 2023-02-13 00:55:44,966 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/3a504ca96acf4c6a9a3dbc93c6381047 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/3a504ca96acf4c6a9a3dbc93c6381047 2023-02-13 00:55:44,966 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f84a259856c34b6695a6bb54160a6af0 of snapshot=snaptb-1676249728028 to region=63f9bed78ea4a19108ec7ecea54182e3 table=testCompactionWithHFileLink 2023-02-13 00:55:44,966 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/36e3c77524c045f1b5bb0e77ef72df16_SeqId_29_ to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/36e3c77524c045f1b5bb0e77ef72df16_SeqId_29_ 2023-02-13 00:55:44,966 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5698639ce41540af9d712c378fe62e38 of snapshot=snaptb-1676249728028 to region=6b485aea7037b0c8ec399d8612e4ff1d table=testCompactionWithHFileLink 2023-02-13 00:55:44,966 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/83ee4d116ee54b588337c72d361fb063 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/83ee4d116ee54b588337c72d361fb063 2023-02-13 00:55:44,966 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink b764582729ff4ae3a7b761976a0a3c41 of snapshot=snaptb-1676249728028 to region=89b71b72a624c785439c7640cd6fb013 table=testCompactionWithHFileLink 2023-02-13 00:55:44,971 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 54c962e475344ba18eb8b404700e643d of snapshot=snaptb-1676249728028 to region=63f9bed78ea4a19108ec7ecea54182e3 table=testCompactionWithHFileLink 2023-02-13 00:55:44,971 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 0ed061c653df44ce9f169ddc4b5dfa54 of snapshot=snaptb-1676249728028 to region=6b485aea7037b0c8ec399d8612e4ff1d table=testCompactionWithHFileLink 2023-02-13 00:55:44,972 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 42284400d5f9487b972af1c61186b106 of snapshot=snaptb-1676249728028 to region=89b71b72a624c785439c7640cd6fb013 table=testCompactionWithHFileLink 2023-02-13 00:55:44,975 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink fd053da68487426db3e6dcbdb9604237 of snapshot=snaptb-1676249728028 to region=63f9bed78ea4a19108ec7ecea54182e3 table=testCompactionWithHFileLink 2023-02-13 00:55:44,976 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d0d7911c42c84855967dab4d508097d9 of snapshot=snaptb-1676249728028 to region=6b485aea7037b0c8ec399d8612e4ff1d table=testCompactionWithHFileLink 2023-02-13 00:55:44,977 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink fd70930d268d4ba6afdcdacce99ab021 of snapshot=snaptb-1676249728028 to region=89b71b72a624c785439c7640cd6fb013 table=testCompactionWithHFileLink 2023-02-13 00:55:44,979 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 61f19ef16aa843998ec2e2f36c298aa1 of snapshot=snaptb-1676249728028 to region=63f9bed78ea4a19108ec7ecea54182e3 table=testCompactionWithHFileLink 2023-02-13 00:55:44,979 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 85892d97287240ffb900a29077361472 of snapshot=snaptb-1676249728028 to region=6b485aea7037b0c8ec399d8612e4ff1d table=testCompactionWithHFileLink 2023-02-13 00:55:44,981 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 04385aa81e7440e7a4d326422c60873f of snapshot=snaptb-1676249728028 to region=89b71b72a624c785439c7640cd6fb013 table=testCompactionWithHFileLink 2023-02-13 00:55:44,990 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/01efca4802b34dc8baf2986ca31e146b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/01efca4802b34dc8baf2986ca31e146b 2023-02-13 00:55:44,990 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 40cc4bcf9b0e42b18fdb316683d81e79 of snapshot=snaptb-1676249728028 to region=63f9bed78ea4a19108ec7ecea54182e3 table=testCompactionWithHFileLink 2023-02-13 00:55:44,990 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/400ea6b3334b476294cee24f0da79ff9 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/400ea6b3334b476294cee24f0da79ff9 2023-02-13 00:55:44,990 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c012deef21c64190a31b6c1d6cd799b4 of snapshot=snaptb-1676249728028 to region=6b485aea7037b0c8ec399d8612e4ff1d table=testCompactionWithHFileLink 2023-02-13 00:55:44,992 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/fa26ce03ae034f1e84d98fe6b148ca16 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/fa26ce03ae034f1e84d98fe6b148ca16 2023-02-13 00:55:44,992 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a0f7d35e413a49d6bf0acc6f8d6693b6 of snapshot=snaptb-1676249728028 to region=89b71b72a624c785439c7640cd6fb013 table=testCompactionWithHFileLink 2023-02-13 00:55:44,996 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink b23fab6f21d54fc79bf2d85aaa21d98d of snapshot=snaptb-1676249728028 to region=63f9bed78ea4a19108ec7ecea54182e3 table=testCompactionWithHFileLink 2023-02-13 00:55:44,996 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d56089120eee43ac9a8f4bee550a86d0 of snapshot=snaptb-1676249728028 to region=6b485aea7037b0c8ec399d8612e4ff1d table=testCompactionWithHFileLink 2023-02-13 00:55:44,998 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 8b40a4cff14a42e1b4f7b39a7371f146 of snapshot=snaptb-1676249728028 to region=89b71b72a624c785439c7640cd6fb013 table=testCompactionWithHFileLink 2023-02-13 00:55:45,000 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ba1edce2256940b0ab26d5d2235ebf4e of snapshot=snaptb-1676249728028 to region=63f9bed78ea4a19108ec7ecea54182e3 table=testCompactionWithHFileLink 2023-02-13 00:55:45,000 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9124faf925464c62955abcbcf2b28843 of snapshot=snaptb-1676249728028 to region=6b485aea7037b0c8ec399d8612e4ff1d table=testCompactionWithHFileLink 2023-02-13 00:55:45,001 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5aa6bed313e74ae0b57bb6aba6db12aa of snapshot=snaptb-1676249728028 to region=89b71b72a624c785439c7640cd6fb013 table=testCompactionWithHFileLink 2023-02-13 00:55:45,003 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d095b833edb04e21a0f6de413b1e43f9 of snapshot=snaptb-1676249728028 to region=6b485aea7037b0c8ec399d8612e4ff1d table=testCompactionWithHFileLink 2023-02-13 00:55:45,003 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ad7c82764fe141a49defe528f6bd7f78 of snapshot=snaptb-1676249728028 to region=63f9bed78ea4a19108ec7ecea54182e3 table=testCompactionWithHFileLink 2023-02-13 00:55:45,005 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink b136ea849d0e4e6a8cbe2bb45e6ac4b8 of snapshot=snaptb-1676249728028 to region=89b71b72a624c785439c7640cd6fb013 table=testCompactionWithHFileLink 2023-02-13 00:55:45,008 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1676249728028" table: "testCompactionWithHFileLink" creation_time: 1676249728038 type: FLUSH version: 2 ttl: 0 2023-02-13 00:55:45,009 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249745009"}]},"ts":"9223372036854775807"} 2023-02-13 00:55:45,010 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249745009"}]},"ts":"9223372036854775807"} 2023-02-13 00:55:45,010 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249745009"}]},"ts":"9223372036854775807"} 2023-02-13 00:55:45,011 INFO [PEWorker-5] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 00:55:45,012 DEBUG [PEWorker-5] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 63f9bed78ea4a19108ec7ecea54182e3, NAME => 'testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => 6b485aea7037b0c8ec399d8612e4ff1d, NAME => 'testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => 89b71b72a624c785439c7640cd6fb013, NAME => 'testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013.', STARTKEY => '', ENDKEY => '1'}] 2023-02-13 00:55:45,012 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676249745010"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249745010"}]},"ts":"1676249745010"} 2023-02-13 00:55:45,012 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676249745010"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249745010"}]},"ts":"1676249745010"} 2023-02-13 00:55:45,012 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676249745010"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249745010"}]},"ts":"1676249745010"} 2023-02-13 00:55:45,013 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-13 00:55:45,013 INFO [PEWorker-5] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-02-13 00:55:45,013 DEBUG [PEWorker-5] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 63f9bed78ea4a19108ec7ecea54182e3, NAME => 'testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => 6b485aea7037b0c8ec399d8612e4ff1d, NAME => 'testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => 89b71b72a624c785439c7640cd6fb013, NAME => 'testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013.', STARTKEY => '', ENDKEY => '1'}] 2023-02-13 00:55:45,014 INFO [PEWorker-5] procedure.RestoreSnapshotProcedure(447): Restore snapshot={ ss=snaptb-1676249728028 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-02-13 00:55:45,015 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1676249728028" table: "testCompactionWithHFileLink" creation_time: 1676249728038 type: FLUSH version: 2 ttl: 0 ) in 526 msec 2023-02-13 00:55:45,102 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 00:55:45,102 INFO [Listener at localhost/37869] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-02-13 00:55:45,102 INFO [Listener at localhost/37869] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1676249728028-1676249742337 2023-02-13 00:55:45,106 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(748): Client=jenkins//172.31.14.131 delete name: "hbase-failsafe-snaptb-1676249728028-1676249742337" 2023-02-13 00:55:45,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] snapshot.SnapshotManager(333): Deleting snapshot: hbase-failsafe-snaptb-1676249728028-1676249742337 2023-02-13 00:55:45,111 INFO [Listener at localhost/37869] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-02-13 00:55:45,115 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.HMaster$10(2372): Client=jenkins//172.31.14.131 enable testCompactionWithHFileLink 2023-02-13 00:55:45,120 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure2.ProcedureExecutor(1029): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-02-13 00:55:45,122 INFO [PEWorker-3] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-02-13 00:55:45,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-13 00:55:45,123 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249745122"}]},"ts":"1676249745122"} 2023-02-13 00:55:45,124 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-02-13 00:55:45,126 DEBUG [PEWorker-3] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-02-13 00:55:45,126 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=89b71b72a624c785439c7640cd6fb013, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6b485aea7037b0c8ec399d8612e4ff1d, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=63f9bed78ea4a19108ec7ecea54182e3, ASSIGN}] 2023-02-13 00:55:45,127 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=89b71b72a624c785439c7640cd6fb013, ASSIGN 2023-02-13 00:55:45,127 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=63f9bed78ea4a19108ec7ecea54182e3, ASSIGN 2023-02-13 00:55:45,127 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6b485aea7037b0c8ec399d8612e4ff1d, ASSIGN 2023-02-13 00:55:45,128 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=89b71b72a624c785439c7640cd6fb013, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-13 00:55:45,128 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=63f9bed78ea4a19108ec7ecea54182e3, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-13 00:55:45,128 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6b485aea7037b0c8ec399d8612e4ff1d, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-13 00:55:45,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-13 00:55:45,279 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=6b485aea7037b0c8ec399d8612e4ff1d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:45,279 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=63f9bed78ea4a19108ec7ecea54182e3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:45,279 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=89b71b72a624c785439c7640cd6fb013, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:45,279 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676249745279"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249745279"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249745279"}]},"ts":"1676249745279"} 2023-02-13 00:55:45,280 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676249745279"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249745279"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249745279"}]},"ts":"1676249745279"} 2023-02-13 00:55:45,280 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676249745279"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249745279"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249745279"}]},"ts":"1676249745279"} 2023-02-13 00:55:45,281 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=101, ppid=99, state=RUNNABLE; OpenRegionProcedure 6b485aea7037b0c8ec399d8612e4ff1d, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:55:45,282 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=100, state=RUNNABLE; OpenRegionProcedure 63f9bed78ea4a19108ec7ecea54182e3, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:55:45,282 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=98, state=RUNNABLE; OpenRegionProcedure 89b71b72a624c785439c7640cd6fb013, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:55:45,424 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-13 00:55:45,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:45,435 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 89b71b72a624c785439c7640cd6fb013, NAME => 'testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013.', STARTKEY => '', ENDKEY => '1'} 2023-02-13 00:55:45,435 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 89b71b72a624c785439c7640cd6fb013 2023-02-13 00:55:45,435 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:45,435 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 89b71b72a624c785439c7640cd6fb013 2023-02-13 00:55:45,435 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 89b71b72a624c785439c7640cd6fb013 2023-02-13 00:55:45,436 INFO [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 89b71b72a624c785439c7640cd6fb013 2023-02-13 00:55:45,437 DEBUG [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1 2023-02-13 00:55:45,437 DEBUG [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1 2023-02-13 00:55:45,437 INFO [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 89b71b72a624c785439c7640cd6fb013 columnFamilyName family1 2023-02-13 00:55:45,448 DEBUG [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-04385aa81e7440e7a4d326422c60873f 2023-02-13 00:55:45,453 DEBUG [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-42284400d5f9487b972af1c61186b106 2023-02-13 00:55:45,458 DEBUG [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-b764582729ff4ae3a7b761976a0a3c41 2023-02-13 00:55:45,463 DEBUG [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-fd70930d268d4ba6afdcdacce99ab021 2023-02-13 00:55:45,463 INFO [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] regionserver.HStore(310): Store=89b71b72a624c785439c7640cd6fb013/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:45,464 INFO [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 89b71b72a624c785439c7640cd6fb013 2023-02-13 00:55:45,464 DEBUG [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2 2023-02-13 00:55:45,464 DEBUG [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2 2023-02-13 00:55:45,465 INFO [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 89b71b72a624c785439c7640cd6fb013 columnFamilyName family2 2023-02-13 00:55:45,471 DEBUG [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-5aa6bed313e74ae0b57bb6aba6db12aa 2023-02-13 00:55:45,475 DEBUG [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-8b40a4cff14a42e1b4f7b39a7371f146 2023-02-13 00:55:45,480 DEBUG [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-a0f7d35e413a49d6bf0acc6f8d6693b6 2023-02-13 00:55:45,485 DEBUG [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-b136ea849d0e4e6a8cbe2bb45e6ac4b8 2023-02-13 00:55:45,485 INFO [StoreOpener-89b71b72a624c785439c7640cd6fb013-1] regionserver.HStore(310): Store=89b71b72a624c785439c7640cd6fb013/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:45,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013 2023-02-13 00:55:45,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013 2023-02-13 00:55:45,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:55:45,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 89b71b72a624c785439c7640cd6fb013 2023-02-13 00:55:45,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 89b71b72a624c785439c7640cd6fb013; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70213756, jitterRate=0.04626649618148804}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:55:45,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 89b71b72a624c785439c7640cd6fb013: Running coprocessor pre-open hook at 1676249745435Writing region info on filesystem at 1676249745435Initializing all the Stores at 1676249745436 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249745436Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249745436Cleaning up temporary data from old regions at 1676249745487 (+51 ms)Cleaning up detritus from prior splits at 1676249745487Running coprocessor post-open hooks at 1676249745489 (+2 ms)Region opened successfully at 1676249745489 2023-02-13 00:55:45,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013., pid=103, masterSystemTime=1676249745432 2023-02-13 00:55:45,490 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:55:45,490 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 00:55:45,490 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 00:55:45,490 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 00:55:45,492 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:45,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:45,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:45,492 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 63f9bed78ea4a19108ec7ecea54182e3, NAME => 'testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3.', STARTKEY => '2', ENDKEY => ''} 2023-02-13 00:55:45,492 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=89b71b72a624c785439c7640cd6fb013, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:45,492 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 63f9bed78ea4a19108ec7ecea54182e3 2023-02-13 00:55:45,492 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676249745492"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249745492"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249745492"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249745492"}]},"ts":"1676249745492"} 2023-02-13 00:55:45,492 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:45,492 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 63f9bed78ea4a19108ec7ecea54182e3 2023-02-13 00:55:45,492 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 63f9bed78ea4a19108ec7ecea54182e3 2023-02-13 00:55:45,493 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-13 00:55:45,493 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-13 00:55:45,493 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] regionserver.HStore(1912): 89b71b72a624c785439c7640cd6fb013/family2 is initiating minor compaction (all files) 2023-02-13 00:55:45,493 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1912): 89b71b72a624c785439c7640cd6fb013/family1 is initiating minor compaction (all files) 2023-02-13 00:55:45,493 INFO [RS:0;jenkins-hbase4:36605-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 89b71b72a624c785439c7640cd6fb013/family2 in testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:45,493 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 89b71b72a624c785439c7640cd6fb013/family1 in testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:55:45,493 INFO [RS:0;jenkins-hbase4:36605-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-a0f7d35e413a49d6bf0acc6f8d6693b6, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-8b40a4cff14a42e1b4f7b39a7371f146, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-5aa6bed313e74ae0b57bb6aba6db12aa, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-b136ea849d0e4e6a8cbe2bb45e6ac4b8] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp, totalSize=20.4 K 2023-02-13 00:55:45,493 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-b764582729ff4ae3a7b761976a0a3c41, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-42284400d5f9487b972af1c61186b106, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-fd70930d268d4ba6afdcdacce99ab021, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-04385aa81e7440e7a4d326422c60873f] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp, totalSize=21.6 K 2023-02-13 00:55:45,494 INFO [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 63f9bed78ea4a19108ec7ecea54182e3 2023-02-13 00:55:45,494 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-a0f7d35e413a49d6bf0acc6f8d6693b6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676249720883 2023-02-13 00:55:45,494 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-b764582729ff4ae3a7b761976a0a3c41, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676249720883 2023-02-13 00:55:45,495 DEBUG [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1 2023-02-13 00:55:45,495 DEBUG [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1 2023-02-13 00:55:45,495 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-8b40a4cff14a42e1b4f7b39a7371f146, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676249722686 2023-02-13 00:55:45,495 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=98 2023-02-13 00:55:45,495 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-42284400d5f9487b972af1c61186b106, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676249722686 2023-02-13 00:55:45,495 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=98, state=SUCCESS; OpenRegionProcedure 89b71b72a624c785439c7640cd6fb013, server=jenkins-hbase4.apache.org,36605,1676249622318 in 212 msec 2023-02-13 00:55:45,495 INFO [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 63f9bed78ea4a19108ec7ecea54182e3 columnFamilyName family1 2023-02-13 00:55:45,496 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-5aa6bed313e74ae0b57bb6aba6db12aa, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676249724465 2023-02-13 00:55:45,496 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-fd70930d268d4ba6afdcdacce99ab021, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676249724465 2023-02-13 00:55:45,496 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=89b71b72a624c785439c7640cd6fb013, ASSIGN in 369 msec 2023-02-13 00:55:45,496 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-b136ea849d0e4e6a8cbe2bb45e6ac4b8, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676249726252 2023-02-13 00:55:45,496 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-04385aa81e7440e7a4d326422c60873f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676249726252 2023-02-13 00:55:45,508 DEBUG [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-54c962e475344ba18eb8b404700e643d 2023-02-13 00:55:45,516 DEBUG [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-61f19ef16aa843998ec2e2f36c298aa1 2023-02-13 00:55:45,534 DEBUG [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-f84a259856c34b6695a6bb54160a6af0 2023-02-13 00:55:45,541 INFO [RS:0;jenkins-hbase4:36605-longCompactions-0] throttle.PressureAwareThroughputController(145): 89b71b72a624c785439c7640cd6fb013#family2#compaction#113 average throughput is 0.08 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:55:45,544 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] throttle.PressureAwareThroughputController(145): 89b71b72a624c785439c7640cd6fb013#family1#compaction#114 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:55:45,551 DEBUG [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-fd053da68487426db3e6dcbdb9604237 2023-02-13 00:55:45,556 INFO [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] regionserver.HStore(310): Store=63f9bed78ea4a19108ec7ecea54182e3/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:45,556 INFO [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 63f9bed78ea4a19108ec7ecea54182e3 2023-02-13 00:55:45,557 DEBUG [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2 2023-02-13 00:55:45,557 DEBUG [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2 2023-02-13 00:55:45,558 INFO [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 63f9bed78ea4a19108ec7ecea54182e3 columnFamilyName family2 2023-02-13 00:55:45,573 DEBUG [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-40cc4bcf9b0e42b18fdb316683d81e79 2023-02-13 00:55:45,577 DEBUG [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-ad7c82764fe141a49defe528f6bd7f78 2023-02-13 00:55:45,582 DEBUG [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-b23fab6f21d54fc79bf2d85aaa21d98d 2023-02-13 00:55:45,589 DEBUG [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-ba1edce2256940b0ab26d5d2235ebf4e 2023-02-13 00:55:45,589 INFO [StoreOpener-63f9bed78ea4a19108ec7ecea54182e3-1] regionserver.HStore(310): Store=63f9bed78ea4a19108ec7ecea54182e3/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:45,590 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3 2023-02-13 00:55:45,591 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3 2023-02-13 00:55:45,593 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:55:45,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 63f9bed78ea4a19108ec7ecea54182e3 2023-02-13 00:55:45,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 63f9bed78ea4a19108ec7ecea54182e3; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70918021, jitterRate=0.05676086246967316}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:55:45,595 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 63f9bed78ea4a19108ec7ecea54182e3: Running coprocessor pre-open hook at 1676249745493Writing region info on filesystem at 1676249745493Initializing all the Stores at 1676249745493Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249745494 (+1 ms)Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249745494Cleaning up temporary data from old regions at 1676249745591 (+97 ms)Cleaning up detritus from prior splits at 1676249745592 (+1 ms)Running coprocessor post-open hooks at 1676249745595 (+3 ms)Region opened successfully at 1676249745595 2023-02-13 00:55:45,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3., pid=102, masterSystemTime=1676249745432 2023-02-13 00:55:45,596 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 00:55:45,596 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-13 00:55:45,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:45,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:45,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:45,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6b485aea7037b0c8ec399d8612e4ff1d, NAME => 'testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d.', STARTKEY => '1', ENDKEY => '2'} 2023-02-13 00:55:45,597 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=63f9bed78ea4a19108ec7ecea54182e3, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:45,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 6b485aea7037b0c8ec399d8612e4ff1d 2023-02-13 00:55:45,597 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676249745597"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249745597"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249745597"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249745597"}]},"ts":"1676249745597"} 2023-02-13 00:55:45,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:45,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 6b485aea7037b0c8ec399d8612e4ff1d 2023-02-13 00:55:45,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 6b485aea7037b0c8ec399d8612e4ff1d 2023-02-13 00:55:45,599 INFO [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6b485aea7037b0c8ec399d8612e4ff1d 2023-02-13 00:55:45,600 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=100 2023-02-13 00:55:45,600 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=100, state=SUCCESS; OpenRegionProcedure 63f9bed78ea4a19108ec7ecea54182e3, server=jenkins-hbase4.apache.org,36605,1676249622318 in 317 msec 2023-02-13 00:55:45,600 DEBUG [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1 2023-02-13 00:55:45,601 DEBUG [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1 2023-02-13 00:55:45,601 INFO [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6b485aea7037b0c8ec399d8612e4ff1d columnFamilyName family1 2023-02-13 00:55:45,601 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=63f9bed78ea4a19108ec7ecea54182e3, ASSIGN in 474 msec 2023-02-13 00:55:45,608 DEBUG [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-0ed061c653df44ce9f169ddc4b5dfa54 2023-02-13 00:55:45,615 DEBUG [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-5698639ce41540af9d712c378fe62e38 2023-02-13 00:55:45,622 DEBUG [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-85892d97287240ffb900a29077361472 2023-02-13 00:55:45,628 DEBUG [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-d0d7911c42c84855967dab4d508097d9 2023-02-13 00:55:45,628 INFO [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] regionserver.HStore(310): Store=6b485aea7037b0c8ec399d8612e4ff1d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:45,629 INFO [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 6b485aea7037b0c8ec399d8612e4ff1d 2023-02-13 00:55:45,629 DEBUG [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2 2023-02-13 00:55:45,629 DEBUG [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2 2023-02-13 00:55:45,630 INFO [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6b485aea7037b0c8ec399d8612e4ff1d columnFamilyName family2 2023-02-13 00:55:45,637 DEBUG [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-9124faf925464c62955abcbcf2b28843 2023-02-13 00:55:45,644 DEBUG [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-c012deef21c64190a31b6c1d6cd799b4 2023-02-13 00:55:45,650 DEBUG [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-d095b833edb04e21a0f6de413b1e43f9 2023-02-13 00:55:45,655 DEBUG [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] regionserver.HStore(539): loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-d56089120eee43ac9a8f4bee550a86d0 2023-02-13 00:55:45,655 INFO [StoreOpener-6b485aea7037b0c8ec399d8612e4ff1d-1] regionserver.HStore(310): Store=6b485aea7037b0c8ec399d8612e4ff1d/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:45,656 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d 2023-02-13 00:55:45,657 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d 2023-02-13 00:55:45,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:55:45,659 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 6b485aea7037b0c8ec399d8612e4ff1d 2023-02-13 00:55:45,660 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 6b485aea7037b0c8ec399d8612e4ff1d; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72443871, jitterRate=0.0794977992773056}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:55:45,660 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 6b485aea7037b0c8ec399d8612e4ff1d: Running coprocessor pre-open hook at 1676249745598Writing region info on filesystem at 1676249745598Initializing all the Stores at 1676249745598Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249745599 (+1 ms)Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249745599Cleaning up temporary data from old regions at 1676249745658 (+59 ms)Cleaning up detritus from prior splits at 1676249745658Running coprocessor post-open hooks at 1676249745660 (+2 ms)Region opened successfully at 1676249745660 2023-02-13 00:55:45,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d., pid=101, masterSystemTime=1676249745432 2023-02-13 00:55:45,661 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-02-13 00:55:45,661 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-13 00:55:45,662 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:45,662 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:45,663 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=6b485aea7037b0c8ec399d8612e4ff1d, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:45,663 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676249745663"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249745663"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249745663"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249745663"}]},"ts":"1676249745663"} 2023-02-13 00:55:45,665 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=101, resume processing ppid=99 2023-02-13 00:55:45,665 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=101, ppid=99, state=SUCCESS; OpenRegionProcedure 6b485aea7037b0c8ec399d8612e4ff1d, server=jenkins-hbase4.apache.org,36605,1676249622318 in 383 msec 2023-02-13 00:55:45,667 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=97 2023-02-13 00:55:45,667 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6b485aea7037b0c8ec399d8612e4ff1d, ASSIGN in 539 msec 2023-02-13 00:55:45,667 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249745667"}]},"ts":"1676249745667"} 2023-02-13 00:55:45,668 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-02-13 00:55:45,670 INFO [PEWorker-4] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-02-13 00:55:45,671 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 554 msec 2023-02-13 00:55:45,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-13 00:55:45,725 INFO [Listener at localhost/37869] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-02-13 00:55:45,738 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 00:55:45,748 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 00:55:45,757 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 00:55:45,758 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 00:55:45,765 INFO [Listener at localhost/37869] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-13 00:55:45,784 INFO [Listener at localhost/37869] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-02-13 00:55:45,784 INFO [Listener at localhost/37869] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-02-13 00:55:45,784 DEBUG [Listener at localhost/37869] zookeeper.ReadOnlyZKClient(139): Connect 0x6a875512 to 127.0.0.1:54478 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:55:45,789 DEBUG [Listener at localhost/37869] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@18ecd160, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:55:45,789 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 00:55:45,789 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 00:55:45,790 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 00:55:46,378 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp/family1/05cc16462f4846c7936e907aef3b99bd as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/05cc16462f4846c7936e907aef3b99bd 2023-02-13 00:55:46,378 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/.tmp/family2/06ca88d354ab4dafb5a89c6cbfd94ba3 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/06ca88d354ab4dafb5a89c6cbfd94ba3 2023-02-13 00:55:46,382 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-02-13 00:55:46,382 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-02-13 00:55:46,384 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 89b71b72a624c785439c7640cd6fb013/family1 of 89b71b72a624c785439c7640cd6fb013 into 05cc16462f4846c7936e907aef3b99bd(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:55:46,384 INFO [RS:0;jenkins-hbase4:36605-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 89b71b72a624c785439c7640cd6fb013/family2 of 89b71b72a624c785439c7640cd6fb013 into 06ca88d354ab4dafb5a89c6cbfd94ba3(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:55:46,384 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 89b71b72a624c785439c7640cd6fb013: 2023-02-13 00:55:46,384 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 89b71b72a624c785439c7640cd6fb013: 2023-02-13 00:55:46,384 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013., storeName=89b71b72a624c785439c7640cd6fb013/family1, priority=12, startTime=1676249745490; duration=0sec 2023-02-13 00:55:46,384 INFO [RS:0;jenkins-hbase4:36605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013., storeName=89b71b72a624c785439c7640cd6fb013/family2, priority=12, startTime=1676249745490; duration=0sec 2023-02-13 00:55:46,384 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-13 00:55:46,384 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-02-13 00:55:46,384 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 00:55:46,385 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 00:55:46,387 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-13 00:55:46,387 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-13 00:55:46,387 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1912): 63f9bed78ea4a19108ec7ecea54182e3/family1 is initiating minor compaction (all files) 2023-02-13 00:55:46,387 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] regionserver.HStore(1912): 6b485aea7037b0c8ec399d8612e4ff1d/family2 is initiating minor compaction (all files) 2023-02-13 00:55:46,387 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 63f9bed78ea4a19108ec7ecea54182e3/family1 in testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:46,387 INFO [RS:0;jenkins-hbase4:36605-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 6b485aea7037b0c8ec399d8612e4ff1d/family2 in testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:46,387 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-f84a259856c34b6695a6bb54160a6af0, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-54c962e475344ba18eb8b404700e643d, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-fd053da68487426db3e6dcbdb9604237, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-61f19ef16aa843998ec2e2f36c298aa1] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp, totalSize=21.6 K 2023-02-13 00:55:46,387 INFO [RS:0;jenkins-hbase4:36605-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-c012deef21c64190a31b6c1d6cd799b4, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-d56089120eee43ac9a8f4bee550a86d0, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-9124faf925464c62955abcbcf2b28843, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-d095b833edb04e21a0f6de413b1e43f9] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp, totalSize=20.4 K 2023-02-13 00:55:46,387 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-f84a259856c34b6695a6bb54160a6af0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676249720885 2023-02-13 00:55:46,387 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-c012deef21c64190a31b6c1d6cd799b4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676249720884 2023-02-13 00:55:46,388 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-54c962e475344ba18eb8b404700e643d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676249722687 2023-02-13 00:55:46,388 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-d56089120eee43ac9a8f4bee550a86d0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676249722686 2023-02-13 00:55:46,388 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-fd053da68487426db3e6dcbdb9604237, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676249724465 2023-02-13 00:55:46,388 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-9124faf925464c62955abcbcf2b28843, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676249724465 2023-02-13 00:55:46,389 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-61f19ef16aa843998ec2e2f36c298aa1, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676249726252 2023-02-13 00:55:46,389 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-d095b833edb04e21a0f6de413b1e43f9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676249726252 2023-02-13 00:55:46,400 INFO [RS:0;jenkins-hbase4:36605-longCompactions-0] throttle.PressureAwareThroughputController(145): 6b485aea7037b0c8ec399d8612e4ff1d#family2#compaction#115 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:55:46,401 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] throttle.PressureAwareThroughputController(145): 63f9bed78ea4a19108ec7ecea54182e3#family1#compaction#116 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:55:46,436 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-b764582729ff4ae3a7b761976a0a3c41, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-42284400d5f9487b972af1c61186b106, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-fd70930d268d4ba6afdcdacce99ab021, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-04385aa81e7440e7a4d326422c60873f] to archive 2023-02-13 00:55:46,437 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:46,438 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-b764582729ff4ae3a7b761976a0a3c41 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-b764582729ff4ae3a7b761976a0a3c41 2023-02-13 00:55:46,439 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-42284400d5f9487b972af1c61186b106 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-42284400d5f9487b972af1c61186b106 2023-02-13 00:55:46,439 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-fd70930d268d4ba6afdcdacce99ab021 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-fd70930d268d4ba6afdcdacce99ab021 2023-02-13 00:55:46,440 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-04385aa81e7440e7a4d326422c60873f to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-04385aa81e7440e7a4d326422c60873f 2023-02-13 00:55:46,441 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-a0f7d35e413a49d6bf0acc6f8d6693b6, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-8b40a4cff14a42e1b4f7b39a7371f146, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-5aa6bed313e74ae0b57bb6aba6db12aa, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-b136ea849d0e4e6a8cbe2bb45e6ac4b8] to archive 2023-02-13 00:55:46,442 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:46,443 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-a0f7d35e413a49d6bf0acc6f8d6693b6 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-a0f7d35e413a49d6bf0acc6f8d6693b6 2023-02-13 00:55:46,443 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-8b40a4cff14a42e1b4f7b39a7371f146 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-8b40a4cff14a42e1b4f7b39a7371f146 2023-02-13 00:55:46,444 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-5aa6bed313e74ae0b57bb6aba6db12aa to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-5aa6bed313e74ae0b57bb6aba6db12aa 2023-02-13 00:55:46,445 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-b136ea849d0e4e6a8cbe2bb45e6ac4b8 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family2/testCompactionWithHFileLink=89b71b72a624c785439c7640cd6fb013-b136ea849d0e4e6a8cbe2bb45e6ac4b8 2023-02-13 00:55:46,682 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213f5d85d4d657d4e659467a543d9a60733 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230213f5d85d4d657d4e659467a543d9a60733 2023-02-13 00:55:46,683 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 00:55:46,687 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213da4d398ea34b42f2977ac7a9dffca5b3 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213da4d398ea34b42f2977ac7a9dffca5b3 2023-02-13 00:55:46,688 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 00:55:46,691 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302136e28bf10d20e4852a9f2eb06c7224820 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302136e28bf10d20e4852a9f2eb06c7224820 2023-02-13 00:55:46,692 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 00:55:46,696 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:55:46,700 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36620, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:55:46,710 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 00:55:46,715 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58262, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 00:55:46,734 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213/family1/3b455065343b432f8e3dc9b108a9e8f7 first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-02-13 00:55:46,735 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213/family1/3b455065343b432f8e3dc9b108a9e8f7}] 2023-02-13 00:55:46,736 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213/family1/029bc985584e4ff2903b75ff0173c890 first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-02-13 00:55:46,736 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=32 for row with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213/family1/029bc985584e4ff2903b75ff0173c890}] 2023-02-13 00:55:46,738 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213/family1/528bdcdeac1f43eaa6ec91205de3e164 first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-02-13 00:55:46,739 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213/family1/528bdcdeac1f43eaa6ec91205de3e164}] 2023-02-13 00:55:46,746 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213/family1/528bdcdeac1f43eaa6ec91205de3e164 for inclusion in 63f9bed78ea4a19108ec7ecea54182e3/family1 2023-02-13 00:55:46,746 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213/family1/3b455065343b432f8e3dc9b108a9e8f7 for inclusion in 6b485aea7037b0c8ec399d8612e4ff1d/family1 2023-02-13 00:55:46,746 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213/family1/029bc985584e4ff2903b75ff0173c890 for inclusion in 89b71b72a624c785439c7640cd6fb013/family1 2023-02-13 00:55:46,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-02-13 00:55:46,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-13 00:55:46,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-02-13 00:55:46,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for 63f9bed78ea4a19108ec7ecea54182e3: 2023-02-13 00:55:46,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-13 00:55:46,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-02-13 00:55:46,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36605] regionserver.HStore(777): Region bounds: first= last=1 2023-02-13 00:55:46,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for 6b485aea7037b0c8ec399d8612e4ff1d: 2023-02-13 00:55:46,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for 89b71b72a624c785439c7640cd6fb013: 2023-02-13 00:55:46,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213/family1/528bdcdeac1f43eaa6ec91205de3e164 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__aq46m0c1nrhh679fs8ka4pnlfln0sl9en5dervn84a4t3frbaukfcv76fiiebdpt/family1/528bdcdeac1f43eaa6ec91205de3e164 2023-02-13 00:55:46,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213/family1/3b455065343b432f8e3dc9b108a9e8f7 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__a5cq0tkfe51jlc37d9u7vvu63fcej4b5sduopc26jjv1koe7frasdbo18hqjjidn/family1/3b455065343b432f8e3dc9b108a9e8f7 2023-02-13 00:55:46,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213/family1/029bc985584e4ff2903b75ff0173c890 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__5831ao76rh6cqluc71f4aporv294atvr5mve0g4u2h5jh56622k36tjib1r5ldti/family1/029bc985584e4ff2903b75ff0173c890 2023-02-13 00:55:46,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__a5cq0tkfe51jlc37d9u7vvu63fcej4b5sduopc26jjv1koe7frasdbo18hqjjidn/family1/3b455065343b432f8e3dc9b108a9e8f7 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/ea264474805f445289a367a6c6e1de08_SeqId_34_ 2023-02-13 00:55:46,753 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__a5cq0tkfe51jlc37d9u7vvu63fcej4b5sduopc26jjv1koe7frasdbo18hqjjidn/family1/3b455065343b432f8e3dc9b108a9e8f7 into 6b485aea7037b0c8ec399d8612e4ff1d/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/ea264474805f445289a367a6c6e1de08_SeqId_34_ - updating store file list. 2023-02-13 00:55:46,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__aq46m0c1nrhh679fs8ka4pnlfln0sl9en5dervn84a4t3frbaukfcv76fiiebdpt/family1/528bdcdeac1f43eaa6ec91205de3e164 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/fe42c9b9af1a435ca3cf01610e350dab_SeqId_34_ 2023-02-13 00:55:46,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__5831ao76rh6cqluc71f4aporv294atvr5mve0g4u2h5jh56622k36tjib1r5ldti/family1/029bc985584e4ff2903b75ff0173c890 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/0ce2125651394facb0f45015f002d99b_SeqId_36_ 2023-02-13 00:55:46,754 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__aq46m0c1nrhh679fs8ka4pnlfln0sl9en5dervn84a4t3frbaukfcv76fiiebdpt/family1/528bdcdeac1f43eaa6ec91205de3e164 into 63f9bed78ea4a19108ec7ecea54182e3/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/fe42c9b9af1a435ca3cf01610e350dab_SeqId_34_ - updating store file list. 2023-02-13 00:55:46,754 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__5831ao76rh6cqluc71f4aporv294atvr5mve0g4u2h5jh56622k36tjib1r5ldti/family1/029bc985584e4ff2903b75ff0173c890 into 89b71b72a624c785439c7640cd6fb013/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/0ce2125651394facb0f45015f002d99b_SeqId_36_ - updating store file list. 2023-02-13 00:55:46,757 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/fe42c9b9af1a435ca3cf01610e350dab_SeqId_34_ into 63f9bed78ea4a19108ec7ecea54182e3/family1 2023-02-13 00:55:46,757 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__aq46m0c1nrhh679fs8ka4pnlfln0sl9en5dervn84a4t3frbaukfcv76fiiebdpt/family1/528bdcdeac1f43eaa6ec91205de3e164 into 63f9bed78ea4a19108ec7ecea54182e3/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/fe42c9b9af1a435ca3cf01610e350dab_SeqId_34_) 2023-02-13 00:55:46,758 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/0ce2125651394facb0f45015f002d99b_SeqId_36_ into 89b71b72a624c785439c7640cd6fb013/family1 2023-02-13 00:55:46,758 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__5831ao76rh6cqluc71f4aporv294atvr5mve0g4u2h5jh56622k36tjib1r5ldti/family1/029bc985584e4ff2903b75ff0173c890 into 89b71b72a624c785439c7640cd6fb013/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/family1/0ce2125651394facb0f45015f002d99b_SeqId_36_) 2023-02-13 00:55:46,758 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/ea264474805f445289a367a6c6e1de08_SeqId_34_ into 6b485aea7037b0c8ec399d8612e4ff1d/family1 2023-02-13 00:55:46,758 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__a5cq0tkfe51jlc37d9u7vvu63fcej4b5sduopc26jjv1koe7frasdbo18hqjjidn/family1/3b455065343b432f8e3dc9b108a9e8f7 into 6b485aea7037b0c8ec399d8612e4ff1d/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/ea264474805f445289a367a6c6e1de08_SeqId_34_) 2023-02-13 00:55:46,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__aq46m0c1nrhh679fs8ka4pnlfln0sl9en5dervn84a4t3frbaukfcv76fiiebdpt/family1/528bdcdeac1f43eaa6ec91205de3e164 2023-02-13 00:55:46,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__a5cq0tkfe51jlc37d9u7vvu63fcej4b5sduopc26jjv1koe7frasdbo18hqjjidn/family1/3b455065343b432f8e3dc9b108a9e8f7 2023-02-13 00:55:46,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testCompactionWithHFileLink__5831ao76rh6cqluc71f4aporv294atvr5mve0g4u2h5jh56622k36tjib1r5ldti/family1/029bc985584e4ff2903b75ff0173c890 2023-02-13 00:55:46,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 00:55:46,761 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 00:55:46,761 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 00:55:46,762 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:46,763 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:46,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-02-13 00:55:46,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(1912): 6b485aea7037b0c8ec399d8612e4ff1d/family1 is initiating minor compaction (all files) 2023-02-13 00:55:46,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@18dc872e; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-02-13 00:55:46,764 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302136a074e5589c4444cbea4ec51c8c112f7 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302136a074e5589c4444cbea4ec51c8c112f7 2023-02-13 00:55:46,764 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023021308341f081e314a2dae1e3f07ac8a389b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023021308341f081e314a2dae1e3f07ac8a389b 2023-02-13 00:55:46,765 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 00:55:46,765 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230213c7c589f59d5a46e69f929bfe1462e530 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230213c7c589f59d5a46e69f929bfe1462e530 2023-02-13 00:55:46,765 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:46,766 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023021322eec433a1514cdaaa42c42305d30a12 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023021322eec433a1514cdaaa42c42305d30a12 2023-02-13 00:55:46,766 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230213e74bd61456ae4f18ae9a602ef3d7ea75 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230213e74bd61456ae4f18ae9a602ef3d7ea75 2023-02-13 00:55:46,767 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230213348ca6e4353c43259c1482c61f82b4a5 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230213348ca6e4353c43259c1482c61f82b4a5 2023-02-13 00:55:46,767 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302131dc8db5fc31c4457bc825856a90efcb6 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302131dc8db5fc31c4457bc825856a90efcb6 2023-02-13 00:55:46,767 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230213f09c030f2155421aa571ae8f4246407d to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230213f09c030f2155421aa571ae8f4246407d 2023-02-13 00:55:46,767 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-13 00:55:46,767 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213' 2023-02-13 00:55:46,768 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302138f21581a0c4c4b58a0ad086df3e0ae67 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302138f21581a0c4c4b58a0ad086df3e0ae67 2023-02-13 00:55:46,768 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302132917a3ddef6a4d7e847180bde371f97b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302132917a3ddef6a4d7e847180bde371f97b 2023-02-13 00:55:46,768 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 00:55:46,768 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-13 00:55:46,768 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213' 2023-02-13 00:55:46,768 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 00:55:46,769 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302133736609080864ed1af2fda92d4e1087b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302133736609080864ed1af2fda92d4e1087b 2023-02-13 00:55:46,769 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302139ab092340bd14dbc8b723519529420ae to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302139ab092340bd14dbc8b723519529420ae 2023-02-13 00:55:46,769 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-13 00:55:46,769 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213' 2023-02-13 00:55:46,770 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 00:55:46,770 INFO [Listener at localhost/37869] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-02-13 00:55:46,775 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 00:55:46,777 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 00:55:46,779 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 00:55:46,780 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 00:55:46,785 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 00:55:46,786 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 00:55:46,802 INFO [Listener at localhost/37869] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=284 (was 257) - Thread LEAK? -, OpenFileDescriptor=752 (was 649) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=142 (was 194), ProcessCount=167 (was 167), AvailableMemoryMB=3319 (was 3474) 2023-02-13 00:55:46,814 INFO [Listener at localhost/37869] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=284, OpenFileDescriptor=752, MaxFileDescriptor=60000, SystemLoadAverage=142, ProcessCount=167, AvailableMemoryMB=3319 2023-02-13 00:55:46,819 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.HMaster$15(2997): Client=jenkins//172.31.14.131 creating {NAME => 'ns'} 2023-02-13 00:55:46,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-02-13 00:55:46,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-02-13 00:55:46,826 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 00:55:46,828 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 8 msec 2023-02-13 00:55:46,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-02-13 00:55:46,926 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-13 00:55:46,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-02-13 00:55:46,928 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 00:55:46,928 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-02-13 00:55:46,929 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 00:55:46,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 00:55:46,930 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db 2023-02-13 00:55:46,930 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42 2023-02-13 00:55:46,930 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73 2023-02-13 00:55:46,931 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db empty. 2023-02-13 00:55:46,931 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42 empty. 2023-02-13 00:55:46,931 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73 empty. 2023-02-13 00:55:46,931 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db 2023-02-13 00:55:46,931 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42 2023-02-13 00:55:46,931 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73 2023-02-13 00:55:46,931 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-02-13 00:55:47,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 00:55:47,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 00:55:47,338 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-02-13 00:55:47,339 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7671): creating {ENCODED => 914f10fbb08273a073f4cd678112ab73, NAME => 'ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp 2023-02-13 00:55:47,339 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7671): creating {ENCODED => a55e824ba00709d09709c5889f9c7a42, NAME => 'ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp 2023-02-13 00:55:47,339 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7671): creating {ENCODED => 5bae73767db4da59da83135d4e2a56db, NAME => 'ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp 2023-02-13 00:55:47,531 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 00:55:47,620 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp/family2/c1d52ebbd627446493188215347f2341 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/c1d52ebbd627446493188215347f2341 2023-02-13 00:55:47,620 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp/family1/f5541b82c1d34dd2ba61f68bfee6763e as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/f5541b82c1d34dd2ba61f68bfee6763e 2023-02-13 00:55:47,623 INFO [RS:0;jenkins-hbase4:36605-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 6b485aea7037b0c8ec399d8612e4ff1d/family2 of 6b485aea7037b0c8ec399d8612e4ff1d into c1d52ebbd627446493188215347f2341(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 00:55:47,623 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 63f9bed78ea4a19108ec7ecea54182e3/family1 of 63f9bed78ea4a19108ec7ecea54182e3 into f5541b82c1d34dd2ba61f68bfee6763e(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 00:55:47,624 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6b485aea7037b0c8ec399d8612e4ff1d: 2023-02-13 00:55:47,624 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 63f9bed78ea4a19108ec7ecea54182e3: 2023-02-13 00:55:47,624 INFO [RS:0;jenkins-hbase4:36605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d., storeName=6b485aea7037b0c8ec399d8612e4ff1d/family2, priority=12, startTime=1676249745661; duration=1sec 2023-02-13 00:55:47,624 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3., storeName=63f9bed78ea4a19108ec7ecea54182e3/family1, priority=12, startTime=1676249745596; duration=1sec 2023-02-13 00:55:47,624 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-02-13 00:55:47,624 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-13 00:55:47,624 INFO [RS:0;jenkins-hbase4:36605-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 6b485aea7037b0c8ec399d8612e4ff1d/family1 in testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:55:47,624 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 5 compacting, 0 eligible, 16 blocking 2023-02-13 00:55:47,624 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-13 00:55:47,624 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-13 00:55:47,624 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. because compaction request was cancelled 2023-02-13 00:55:47,624 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 00:55:47,624 INFO [RS:0;jenkins-hbase4:36605-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-5698639ce41540af9d712c378fe62e38, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-0ed061c653df44ce9f169ddc4b5dfa54, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-d0d7911c42c84855967dab4d508097d9, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-85892d97287240ffb900a29077361472, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/ea264474805f445289a367a6c6e1de08_SeqId_34_] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp, totalSize=28.9 K 2023-02-13 00:55:47,625 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-5698639ce41540af9d712c378fe62e38, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676249720884 2023-02-13 00:55:47,625 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-0ed061c653df44ce9f169ddc4b5dfa54, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676249722686 2023-02-13 00:55:47,626 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-d0d7911c42c84855967dab4d508097d9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676249724465 2023-02-13 00:55:47,626 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-13 00:55:47,626 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1912): 63f9bed78ea4a19108ec7ecea54182e3/family2 is initiating minor compaction (all files) 2023-02-13 00:55:47,626 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 63f9bed78ea4a19108ec7ecea54182e3/family2 in testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:55:47,626 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-40cc4bcf9b0e42b18fdb316683d81e79, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-b23fab6f21d54fc79bf2d85aaa21d98d, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-ba1edce2256940b0ab26d5d2235ebf4e, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-ad7c82764fe141a49defe528f6bd7f78] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp, totalSize=20.4 K 2023-02-13 00:55:47,626 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-85892d97287240ffb900a29077361472, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676249726252 2023-02-13 00:55:47,626 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] compactions.Compactor(207): Compacting ea264474805f445289a367a6c6e1de08_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1676249720884 2023-02-13 00:55:47,627 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-40cc4bcf9b0e42b18fdb316683d81e79, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676249720885 2023-02-13 00:55:47,627 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-b23fab6f21d54fc79bf2d85aaa21d98d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676249722687 2023-02-13 00:55:47,628 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-ba1edce2256940b0ab26d5d2235ebf4e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676249724465 2023-02-13 00:55:47,628 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-ad7c82764fe141a49defe528f6bd7f78, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676249726252 2023-02-13 00:55:47,636 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-f84a259856c34b6695a6bb54160a6af0, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-54c962e475344ba18eb8b404700e643d, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-fd053da68487426db3e6dcbdb9604237, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-61f19ef16aa843998ec2e2f36c298aa1] to archive 2023-02-13 00:55:47,636 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:47,638 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-f84a259856c34b6695a6bb54160a6af0 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-f84a259856c34b6695a6bb54160a6af0 2023-02-13 00:55:47,639 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-54c962e475344ba18eb8b404700e643d to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-54c962e475344ba18eb8b404700e643d 2023-02-13 00:55:47,639 INFO [RS:0;jenkins-hbase4:36605-longCompactions-0] throttle.PressureAwareThroughputController(145): 6b485aea7037b0c8ec399d8612e4ff1d#family1#compaction#117 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:55:47,639 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] throttle.PressureAwareThroughputController(145): 63f9bed78ea4a19108ec7ecea54182e3#family2#compaction#118 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:55:47,640 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-fd053da68487426db3e6dcbdb9604237 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-fd053da68487426db3e6dcbdb9604237 2023-02-13 00:55:47,641 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-61f19ef16aa843998ec2e2f36c298aa1 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family1/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-61f19ef16aa843998ec2e2f36c298aa1 2023-02-13 00:55:47,642 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-c012deef21c64190a31b6c1d6cd799b4, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-d56089120eee43ac9a8f4bee550a86d0, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-9124faf925464c62955abcbcf2b28843, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-d095b833edb04e21a0f6de413b1e43f9] to archive 2023-02-13 00:55:47,643 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:47,644 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-c012deef21c64190a31b6c1d6cd799b4 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-c012deef21c64190a31b6c1d6cd799b4 2023-02-13 00:55:47,645 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-d56089120eee43ac9a8f4bee550a86d0 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-d56089120eee43ac9a8f4bee550a86d0 2023-02-13 00:55:47,646 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-9124faf925464c62955abcbcf2b28843 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-9124faf925464c62955abcbcf2b28843 2023-02-13 00:55:47,647 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-d095b833edb04e21a0f6de413b1e43f9 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family2/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-d095b833edb04e21a0f6de413b1e43f9 2023-02-13 00:55:47,724 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 00:55:47,751 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:47,751 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:47,751 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1603): Closing 914f10fbb08273a073f4cd678112ab73, disabling compactions & flushes 2023-02-13 00:55:47,751 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. 2023-02-13 00:55:47,751 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:47,751 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. 2023-02-13 00:55:47,752 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1603): Closing a55e824ba00709d09709c5889f9c7a42, disabling compactions & flushes 2023-02-13 00:55:47,751 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1603): Closing 5bae73767db4da59da83135d4e2a56db, disabling compactions & flushes 2023-02-13 00:55:47,752 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. 2023-02-13 00:55:47,752 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. after waiting 0 ms 2023-02-13 00:55:47,752 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. 2023-02-13 00:55:47,752 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. 2023-02-13 00:55:47,752 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. 2023-02-13 00:55:47,752 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. after waiting 0 ms 2023-02-13 00:55:47,752 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. 2023-02-13 00:55:47,752 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1837): Closed ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. 2023-02-13 00:55:47,752 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1837): Closed ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. 2023-02-13 00:55:47,752 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. 2023-02-13 00:55:47,752 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1557): Region close journal for a55e824ba00709d09709c5889f9c7a42: Waiting for close lock at 1676249747752Disabling compacts and flushes for region at 1676249747752Disabling writes for close at 1676249747752Writing region close event to WAL at 1676249747752Closed at 1676249747752 2023-02-13 00:55:47,752 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1557): Region close journal for 914f10fbb08273a073f4cd678112ab73: Waiting for close lock at 1676249747751Disabling compacts and flushes for region at 1676249747751Disabling writes for close at 1676249747752 (+1 ms)Writing region close event to WAL at 1676249747752Closed at 1676249747752 2023-02-13 00:55:47,752 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. after waiting 0 ms 2023-02-13 00:55:47,752 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. 2023-02-13 00:55:47,752 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. 2023-02-13 00:55:47,753 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1557): Region close journal for 5bae73767db4da59da83135d4e2a56db: Waiting for close lock at 1676249747751Disabling compacts and flushes for region at 1676249747751Disabling writes for close at 1676249747752 (+1 ms)Writing region close event to WAL at 1676249747752Closed at 1676249747752 2023-02-13 00:55:47,755 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 00:55:47,755 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1676249747755"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249747755"}]},"ts":"1676249747755"} 2023-02-13 00:55:47,756 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676249747755"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249747755"}]},"ts":"1676249747755"} 2023-02-13 00:55:47,756 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676249747755"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249747755"}]},"ts":"1676249747755"} 2023-02-13 00:55:47,757 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-13 00:55:47,758 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 00:55:47,758 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249747758"}]},"ts":"1676249747758"} 2023-02-13 00:55:47,759 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-02-13 00:55:47,762 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=5bae73767db4da59da83135d4e2a56db, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=a55e824ba00709d09709c5889f9c7a42, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=914f10fbb08273a073f4cd678112ab73, ASSIGN}] 2023-02-13 00:55:47,763 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=914f10fbb08273a073f4cd678112ab73, ASSIGN 2023-02-13 00:55:47,763 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=a55e824ba00709d09709c5889f9c7a42, ASSIGN 2023-02-13 00:55:47,763 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=5bae73767db4da59da83135d4e2a56db, ASSIGN 2023-02-13 00:55:47,764 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=914f10fbb08273a073f4cd678112ab73, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=false 2023-02-13 00:55:47,764 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=a55e824ba00709d09709c5889f9c7a42, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=false 2023-02-13 00:55:47,764 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=5bae73767db4da59da83135d4e2a56db, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=false 2023-02-13 00:55:47,916 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=914f10fbb08273a073f4cd678112ab73, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:47,916 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=5bae73767db4da59da83135d4e2a56db, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:47,916 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=a55e824ba00709d09709c5889f9c7a42, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:47,917 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676249747916"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249747916"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249747916"}]},"ts":"1676249747916"} 2023-02-13 00:55:47,917 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676249747916"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249747916"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249747916"}]},"ts":"1676249747916"} 2023-02-13 00:55:47,917 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1676249747916"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249747916"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249747916"}]},"ts":"1676249747916"} 2023-02-13 00:55:47,918 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=109, ppid=108, state=RUNNABLE; OpenRegionProcedure 914f10fbb08273a073f4cd678112ab73, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:55:47,918 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=106, state=RUNNABLE; OpenRegionProcedure 5bae73767db4da59da83135d4e2a56db, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:55:47,919 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=107, state=RUNNABLE; OpenRegionProcedure a55e824ba00709d09709c5889f9c7a42, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:55:48,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 00:55:48,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. 2023-02-13 00:55:48,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a55e824ba00709d09709c5889f9c7a42, NAME => 'ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42.', STARTKEY => '1', ENDKEY => '2'} 2023-02-13 00:55:48,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction a55e824ba00709d09709c5889f9c7a42 2023-02-13 00:55:48,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:48,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a55e824ba00709d09709c5889f9c7a42 2023-02-13 00:55:48,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a55e824ba00709d09709c5889f9c7a42 2023-02-13 00:55:48,073 INFO [StoreOpener-a55e824ba00709d09709c5889f9c7a42-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a55e824ba00709d09709c5889f9c7a42 2023-02-13 00:55:48,074 DEBUG [StoreOpener-a55e824ba00709d09709c5889f9c7a42-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1 2023-02-13 00:55:48,074 DEBUG [StoreOpener-a55e824ba00709d09709c5889f9c7a42-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1 2023-02-13 00:55:48,075 INFO [StoreOpener-a55e824ba00709d09709c5889f9c7a42-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a55e824ba00709d09709c5889f9c7a42 columnFamilyName family1 2023-02-13 00:55:48,076 INFO [StoreOpener-a55e824ba00709d09709c5889f9c7a42-1] regionserver.HStore(310): Store=a55e824ba00709d09709c5889f9c7a42/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:48,076 INFO [StoreOpener-a55e824ba00709d09709c5889f9c7a42-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region a55e824ba00709d09709c5889f9c7a42 2023-02-13 00:55:48,077 DEBUG [StoreOpener-a55e824ba00709d09709c5889f9c7a42-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2 2023-02-13 00:55:48,077 DEBUG [StoreOpener-a55e824ba00709d09709c5889f9c7a42-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2 2023-02-13 00:55:48,077 INFO [StoreOpener-a55e824ba00709d09709c5889f9c7a42-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a55e824ba00709d09709c5889f9c7a42 columnFamilyName family2 2023-02-13 00:55:48,078 INFO [StoreOpener-a55e824ba00709d09709c5889f9c7a42-1] regionserver.HStore(310): Store=a55e824ba00709d09709c5889f9c7a42/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:48,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42 2023-02-13 00:55:48,079 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42 2023-02-13 00:55:48,080 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:55:48,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a55e824ba00709d09709c5889f9c7a42 2023-02-13 00:55:48,083 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:55:48,083 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a55e824ba00709d09709c5889f9c7a42; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73458604, jitterRate=0.09461849927902222}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:55:48,083 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a55e824ba00709d09709c5889f9c7a42: Running coprocessor pre-open hook at 1676249748072Writing region info on filesystem at 1676249748072Initializing all the Stores at 1676249748073 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249748073Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249748073Cleaning up temporary data from old regions at 1676249748079 (+6 ms)Cleaning up detritus from prior splits at 1676249748080 (+1 ms)Running coprocessor post-open hooks at 1676249748083 (+3 ms)Region opened successfully at 1676249748083 2023-02-13 00:55:48,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42., pid=111, masterSystemTime=1676249748069 2023-02-13 00:55:48,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. 2023-02-13 00:55:48,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. 2023-02-13 00:55:48,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. 2023-02-13 00:55:48,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 914f10fbb08273a073f4cd678112ab73, NAME => 'ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73.', STARTKEY => '2', ENDKEY => ''} 2023-02-13 00:55:48,085 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=a55e824ba00709d09709c5889f9c7a42, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:48,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 914f10fbb08273a073f4cd678112ab73 2023-02-13 00:55:48,085 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1676249748085"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249748085"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249748085"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249748085"}]},"ts":"1676249748085"} 2023-02-13 00:55:48,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:48,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 914f10fbb08273a073f4cd678112ab73 2023-02-13 00:55:48,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 914f10fbb08273a073f4cd678112ab73 2023-02-13 00:55:48,086 INFO [StoreOpener-914f10fbb08273a073f4cd678112ab73-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 914f10fbb08273a073f4cd678112ab73 2023-02-13 00:55:48,087 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=107 2023-02-13 00:55:48,087 DEBUG [StoreOpener-914f10fbb08273a073f4cd678112ab73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1 2023-02-13 00:55:48,087 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=107, state=SUCCESS; OpenRegionProcedure a55e824ba00709d09709c5889f9c7a42, server=jenkins-hbase4.apache.org,36605,1676249622318 in 167 msec 2023-02-13 00:55:48,087 DEBUG [StoreOpener-914f10fbb08273a073f4cd678112ab73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1 2023-02-13 00:55:48,088 INFO [StoreOpener-914f10fbb08273a073f4cd678112ab73-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 914f10fbb08273a073f4cd678112ab73 columnFamilyName family1 2023-02-13 00:55:48,088 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=a55e824ba00709d09709c5889f9c7a42, ASSIGN in 325 msec 2023-02-13 00:55:48,089 INFO [StoreOpener-914f10fbb08273a073f4cd678112ab73-1] regionserver.HStore(310): Store=914f10fbb08273a073f4cd678112ab73/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:48,089 INFO [StoreOpener-914f10fbb08273a073f4cd678112ab73-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 914f10fbb08273a073f4cd678112ab73 2023-02-13 00:55:48,090 DEBUG [StoreOpener-914f10fbb08273a073f4cd678112ab73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2 2023-02-13 00:55:48,090 DEBUG [StoreOpener-914f10fbb08273a073f4cd678112ab73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2 2023-02-13 00:55:48,090 INFO [StoreOpener-914f10fbb08273a073f4cd678112ab73-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 914f10fbb08273a073f4cd678112ab73 columnFamilyName family2 2023-02-13 00:55:48,091 INFO [StoreOpener-914f10fbb08273a073f4cd678112ab73-1] regionserver.HStore(310): Store=914f10fbb08273a073f4cd678112ab73/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:48,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73 2023-02-13 00:55:48,092 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73 2023-02-13 00:55:48,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:55:48,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 914f10fbb08273a073f4cd678112ab73 2023-02-13 00:55:48,095 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:55:48,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 914f10fbb08273a073f4cd678112ab73; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63445550, jitterRate=-0.05458763241767883}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:55:48,095 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 914f10fbb08273a073f4cd678112ab73: Running coprocessor pre-open hook at 1676249748085Writing region info on filesystem at 1676249748085Initializing all the Stores at 1676249748086 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249748086Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249748086Cleaning up temporary data from old regions at 1676249748092 (+6 ms)Cleaning up detritus from prior splits at 1676249748092Running coprocessor post-open hooks at 1676249748095 (+3 ms)Region opened successfully at 1676249748095 2023-02-13 00:55:48,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73., pid=109, masterSystemTime=1676249748069 2023-02-13 00:55:48,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. 2023-02-13 00:55:48,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. 2023-02-13 00:55:48,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. 2023-02-13 00:55:48,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 5bae73767db4da59da83135d4e2a56db, NAME => 'ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db.', STARTKEY => '', ENDKEY => '1'} 2023-02-13 00:55:48,097 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=914f10fbb08273a073f4cd678112ab73, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:48,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 5bae73767db4da59da83135d4e2a56db 2023-02-13 00:55:48,097 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676249748097"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249748097"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249748097"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249748097"}]},"ts":"1676249748097"} 2023-02-13 00:55:48,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:48,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 5bae73767db4da59da83135d4e2a56db 2023-02-13 00:55:48,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 5bae73767db4da59da83135d4e2a56db 2023-02-13 00:55:48,098 INFO [StoreOpener-5bae73767db4da59da83135d4e2a56db-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 5bae73767db4da59da83135d4e2a56db 2023-02-13 00:55:48,099 DEBUG [StoreOpener-5bae73767db4da59da83135d4e2a56db-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1 2023-02-13 00:55:48,099 DEBUG [StoreOpener-5bae73767db4da59da83135d4e2a56db-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1 2023-02-13 00:55:48,099 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=109, resume processing ppid=108 2023-02-13 00:55:48,100 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=109, ppid=108, state=SUCCESS; OpenRegionProcedure 914f10fbb08273a073f4cd678112ab73, server=jenkins-hbase4.apache.org,36605,1676249622318 in 180 msec 2023-02-13 00:55:48,100 INFO [StoreOpener-5bae73767db4da59da83135d4e2a56db-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5bae73767db4da59da83135d4e2a56db columnFamilyName family1 2023-02-13 00:55:48,100 INFO [StoreOpener-5bae73767db4da59da83135d4e2a56db-1] regionserver.HStore(310): Store=5bae73767db4da59da83135d4e2a56db/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:48,100 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=914f10fbb08273a073f4cd678112ab73, ASSIGN in 338 msec 2023-02-13 00:55:48,101 INFO [StoreOpener-5bae73767db4da59da83135d4e2a56db-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 5bae73767db4da59da83135d4e2a56db 2023-02-13 00:55:48,101 DEBUG [StoreOpener-5bae73767db4da59da83135d4e2a56db-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2 2023-02-13 00:55:48,101 DEBUG [StoreOpener-5bae73767db4da59da83135d4e2a56db-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2 2023-02-13 00:55:48,102 INFO [StoreOpener-5bae73767db4da59da83135d4e2a56db-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5bae73767db4da59da83135d4e2a56db columnFamilyName family2 2023-02-13 00:55:48,102 INFO [StoreOpener-5bae73767db4da59da83135d4e2a56db-1] regionserver.HStore(310): Store=5bae73767db4da59da83135d4e2a56db/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:48,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db 2023-02-13 00:55:48,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db 2023-02-13 00:55:48,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:55:48,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 5bae73767db4da59da83135d4e2a56db 2023-02-13 00:55:48,107 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:55:48,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 5bae73767db4da59da83135d4e2a56db; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72910296, jitterRate=0.086448073387146}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:55:48,107 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 5bae73767db4da59da83135d4e2a56db: Running coprocessor pre-open hook at 1676249748097Writing region info on filesystem at 1676249748097Initializing all the Stores at 1676249748098 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249748098Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249748098Cleaning up temporary data from old regions at 1676249748104 (+6 ms)Cleaning up detritus from prior splits at 1676249748104Running coprocessor post-open hooks at 1676249748107 (+3 ms)Region opened successfully at 1676249748107 2023-02-13 00:55:48,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db., pid=110, masterSystemTime=1676249748069 2023-02-13 00:55:48,108 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. 2023-02-13 00:55:48,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. 2023-02-13 00:55:48,109 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=5bae73767db4da59da83135d4e2a56db, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:48,109 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676249748109"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249748109"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249748109"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249748109"}]},"ts":"1676249748109"} 2023-02-13 00:55:48,111 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=106 2023-02-13 00:55:48,111 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=106, state=SUCCESS; OpenRegionProcedure 5bae73767db4da59da83135d4e2a56db, server=jenkins-hbase4.apache.org,36605,1676249622318 in 192 msec 2023-02-13 00:55:48,112 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-13 00:55:48,112 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=5bae73767db4da59da83135d4e2a56db, ASSIGN in 349 msec 2023-02-13 00:55:48,112 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 00:55:48,112 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249748112"}]},"ts":"1676249748112"} 2023-02-13 00:55:48,113 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-02-13 00:55:48,117 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 00:55:48,117 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 1.1910 sec 2023-02-13 00:55:48,455 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/.tmp/family2/612bc9c8ae8248b887cb9574f5e37310 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/612bc9c8ae8248b887cb9574f5e37310 2023-02-13 00:55:48,458 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 63f9bed78ea4a19108ec7ecea54182e3/family2 of 63f9bed78ea4a19108ec7ecea54182e3 into 612bc9c8ae8248b887cb9574f5e37310(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:55:48,458 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 63f9bed78ea4a19108ec7ecea54182e3: 2023-02-13 00:55:48,458 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3., storeName=63f9bed78ea4a19108ec7ecea54182e3/family2, priority=12, startTime=1676249745596; duration=0sec 2023-02-13 00:55:48,459 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:55:48,545 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-40cc4bcf9b0e42b18fdb316683d81e79, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-b23fab6f21d54fc79bf2d85aaa21d98d, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-ba1edce2256940b0ab26d5d2235ebf4e, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-ad7c82764fe141a49defe528f6bd7f78] to archive 2023-02-13 00:55:48,545 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:48,547 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-40cc4bcf9b0e42b18fdb316683d81e79 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-40cc4bcf9b0e42b18fdb316683d81e79 2023-02-13 00:55:48,548 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-b23fab6f21d54fc79bf2d85aaa21d98d to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-b23fab6f21d54fc79bf2d85aaa21d98d 2023-02-13 00:55:48,549 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-ba1edce2256940b0ab26d5d2235ebf4e to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-ba1edce2256940b0ab26d5d2235ebf4e 2023-02-13 00:55:48,550 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-ad7c82764fe141a49defe528f6bd7f78 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/family2/testCompactionWithHFileLink=63f9bed78ea4a19108ec7ecea54182e3-ad7c82764fe141a49defe528f6bd7f78 2023-02-13 00:55:48,855 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/.tmp/family1/ba754c65f8b741749512237fd5b933e9 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/ba754c65f8b741749512237fd5b933e9 2023-02-13 00:55:48,858 INFO [RS:0;jenkins-hbase4:36605-longCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in 6b485aea7037b0c8ec399d8612e4ff1d/family1 of 6b485aea7037b0c8ec399d8612e4ff1d into ba754c65f8b741749512237fd5b933e9(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 00:55:48,858 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6b485aea7037b0c8ec399d8612e4ff1d: 2023-02-13 00:55:48,858 INFO [RS:0;jenkins-hbase4:36605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1676249746763; duration=1sec 2023-02-13 00:55:48,859 DEBUG [RS:0;jenkins-hbase4:36605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:55:48,940 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-5698639ce41540af9d712c378fe62e38, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-0ed061c653df44ce9f169ddc4b5dfa54, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-d0d7911c42c84855967dab4d508097d9, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-85892d97287240ffb900a29077361472, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/ea264474805f445289a367a6c6e1de08_SeqId_34_] to archive 2023-02-13 00:55:48,940 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:48,941 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-5698639ce41540af9d712c378fe62e38 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-5698639ce41540af9d712c378fe62e38 2023-02-13 00:55:48,942 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-0ed061c653df44ce9f169ddc4b5dfa54 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-0ed061c653df44ce9f169ddc4b5dfa54 2023-02-13 00:55:48,943 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-d0d7911c42c84855967dab4d508097d9 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-d0d7911c42c84855967dab4d508097d9 2023-02-13 00:55:48,944 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-85892d97287240ffb900a29077361472 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/testCompactionWithHFileLink=6b485aea7037b0c8ec399d8612e4ff1d-85892d97287240ffb900a29077361472 2023-02-13 00:55:48,945 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/ea264474805f445289a367a6c6e1de08_SeqId_34_ to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/family1/ea264474805f445289a367a6c6e1de08_SeqId_34_ 2023-02-13 00:55:49,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 00:55:49,033 INFO [Listener at localhost/37869] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-02-13 00:55:49,039 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:55:49,039 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:55:49,039 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:55:49,041 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:49,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-13 00:55:49,043 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-13 00:55:49,043 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:49,043 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-13 00:55:49,043 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-13 00:55:49,044 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:49,044 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 00:55:49,045 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:49,045 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:49,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:49,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:49,045 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:49,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 00:55:49,045 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:49,046 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:49,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:49,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 00:55:49,046 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-13 00:55:49,046 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-13 00:55:49,046 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-13 00:55:49,046 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:49,046 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-13 00:55:49,046 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. 2023-02-13 00:55:49,046 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. started... 2023-02-13 00:55:49,047 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. 2023-02-13 00:55:49,047 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. started... 2023-02-13 00:55:49,048 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. 2023-02-13 00:55:49,048 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 914f10fbb08273a073f4cd678112ab73 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 00:55:49,047 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 00:55:49,048 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:49,048 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 5bae73767db4da59da83135d4e2a56db 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 00:55:49,048 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. started... 2023-02-13 00:55:49,048 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing a55e824ba00709d09709c5889f9c7a42 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 00:55:49,462 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213d64a499bf9124b2b87df7e9888c59188 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230213d64a499bf9124b2b87df7e9888c59188 2023-02-13 00:55:49,463 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213c7b55ce04a844ebeae367a506c12bdc4 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230213c7b55ce04a844ebeae367a506c12bdc4 2023-02-13 00:55:49,464 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213a9644f83cd8f49bda2d818643903d09b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230213a9644f83cd8f49bda2d818643903d09b 2023-02-13 00:55:49,612 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-02-13 00:55:49,867 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/.tmp/family1/70ec1e114290430287f28713178962f6 2023-02-13 00:55:49,867 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/.tmp/family1/d1e6cd4f51f5432086b939cb3083450d 2023-02-13 00:55:49,869 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/.tmp/family1/ef7baba1791d4087a97eaa0aa6a5ec6d 2023-02-13 00:55:50,287 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213a323a525e1e9495c81f940dd5c2be42e to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230213a323a525e1e9495c81f940dd5c2be42e 2023-02-13 00:55:50,287 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213fd945bb0062a4d05b205ffb7152e3e11 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230213fd945bb0062a4d05b205ffb7152e3e11 2023-02-13 00:55:50,287 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302136f1fb6f78cb446a2840da1eac5225ffc to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202302136f1fb6f78cb446a2840da1eac5225ffc 2023-02-13 00:55:50,692 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/.tmp/family2/53bf28f082504c7c8d8c2b1aa5f9879d 2023-02-13 00:55:50,692 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/.tmp/family2/95dec832c3054bcabcd0399055e2efc3 2023-02-13 00:55:50,692 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/.tmp/family2/1b93672aad624b66b5769229e62e5264 2023-02-13 00:55:50,696 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/.tmp/family1/d1e6cd4f51f5432086b939cb3083450d as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/d1e6cd4f51f5432086b939cb3083450d 2023-02-13 00:55:50,696 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/.tmp/family1/70ec1e114290430287f28713178962f6 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/70ec1e114290430287f28713178962f6 2023-02-13 00:55:50,696 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/.tmp/family1/ef7baba1791d4087a97eaa0aa6a5ec6d as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/ef7baba1791d4087a97eaa0aa6a5ec6d 2023-02-13 00:55:50,699 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/d1e6cd4f51f5432086b939cb3083450d, entries=4, sequenceid=5, filesize=5.4 K 2023-02-13 00:55:50,700 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/70ec1e114290430287f28713178962f6, entries=4, sequenceid=5, filesize=5.4 K 2023-02-13 00:55:50,700 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/ef7baba1791d4087a97eaa0aa6a5ec6d, entries=4, sequenceid=5, filesize=5.4 K 2023-02-13 00:55:50,700 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/.tmp/family2/95dec832c3054bcabcd0399055e2efc3 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/95dec832c3054bcabcd0399055e2efc3 2023-02-13 00:55:50,701 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/.tmp/family2/1b93672aad624b66b5769229e62e5264 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/1b93672aad624b66b5769229e62e5264 2023-02-13 00:55:50,701 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/.tmp/family2/53bf28f082504c7c8d8c2b1aa5f9879d as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/53bf28f082504c7c8d8c2b1aa5f9879d 2023-02-13 00:55:50,704 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/95dec832c3054bcabcd0399055e2efc3, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 00:55:50,704 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/1b93672aad624b66b5769229e62e5264, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 00:55:50,705 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/53bf28f082504c7c8d8c2b1aa5f9879d, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 00:55:50,705 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for a55e824ba00709d09709c5889f9c7a42 in 1657ms, sequenceid=5, compaction requested=false 2023-02-13 00:55:50,705 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 5bae73767db4da59da83135d4e2a56db in 1657ms, sequenceid=5, compaction requested=false 2023-02-13 00:55:50,705 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 914f10fbb08273a073f4cd678112ab73 in 1657ms, sequenceid=5, compaction requested=false 2023-02-13 00:55:50,706 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 5bae73767db4da59da83135d4e2a56db: 2023-02-13 00:55:50,706 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for a55e824ba00709d09709c5889f9c7a42: 2023-02-13 00:55:50,706 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. 2023-02-13 00:55:50,706 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 914f10fbb08273a073f4cd678112ab73: 2023-02-13 00:55:50,706 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 00:55:50,706 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. 2023-02-13 00:55:50,706 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. 2023-02-13 00:55:50,706 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 00:55:50,706 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 00:55:50,706 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 00:55:50,706 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:50,706 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-13 00:55:50,706 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-13 00:55:50,708 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:50,708 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:50,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:50,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:50,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:50,709 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:50,709 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:50,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:50,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:50,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:50,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:50,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:50,710 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-13 00:55:50,710 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ebe743e[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:50,710 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-13 00:55:50,710 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:50,712 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:50,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:50,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:50,712 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-13 00:55:50,712 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-13 00:55:50,712 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:55:50,712 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:50,712 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:50,714 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-13 00:55:50,714 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:50,714 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:50,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:50,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:50,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:50,714 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-13 00:55:50,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:50,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:50,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:50,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:50,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:50,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:50,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:50,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:55:50,717 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-13 00:55:50,717 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-13 00:55:50,717 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:50,717 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:50,717 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-13 00:55:50,717 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:50,719 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:50,719 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:50,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:50,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:50,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:50,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:50,719 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:50,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:50,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:50,719 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:50,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:50,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:50,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:50,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:50,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:50,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:50,721 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:50,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:50,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:50,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:50,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:50,723 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:50,723 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:50,723 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 00:55:50,723 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-13 00:55:50,723 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:50,723 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 00:55:50,723 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:50,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:50,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:50,723 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:50,723 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-13 00:55:50,723 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-13 00:55:50,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:50,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:50,723 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:50,724 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:50,724 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:50,724 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:50,724 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:50,824 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:50,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:50,826 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:55:50,827 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:55:50,827 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:55:50,829 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:50,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-13 00:55:50,831 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-13 00:55:50,831 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-13 00:55:50,831 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:50,831 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-13 00:55:50,831 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-13 00:55:50,831 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:50,831 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 00:55:50,832 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:50,832 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:50,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:50,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:50,832 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:50,832 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:50,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 00:55:50,833 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:50,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:50,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 00:55:50,833 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-13 00:55:50,833 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-13 00:55:50,833 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-13 00:55:50,833 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-13 00:55:50,834 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:50,834 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-13 00:55:50,834 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. 2023-02-13 00:55:50,834 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 00:55:50,834 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. 2023-02-13 00:55:50,834 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. started... 2023-02-13 00:55:50,834 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. started... 2023-02-13 00:55:50,834 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. 2023-02-13 00:55:50,834 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:50,834 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 5bae73767db4da59da83135d4e2a56db 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 00:55:50,834 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. started... 2023-02-13 00:55:50,834 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 914f10fbb08273a073f4cd678112ab73 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 00:55:50,834 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing a55e824ba00709d09709c5889f9c7a42 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 00:55:50,852 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213db483c995f394528aafe395eda84e746 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230213db483c995f394528aafe395eda84e746 2023-02-13 00:55:51,253 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213eb6463f4606e4d1e9e6b36ebe69ec8fb to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230213eb6463f4606e4d1e9e6b36ebe69ec8fb 2023-02-13 00:55:51,253 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213565401b8a5e14aafbf0d53f450540c2b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230213565401b8a5e14aafbf0d53f450540c2b 2023-02-13 00:55:51,256 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/.tmp/family1/a1ba8c22a68240d49802410b2775020e 2023-02-13 00:55:51,658 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/.tmp/family1/f379a72bb27448a19088382d0b086fc4 2023-02-13 00:55:51,658 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/.tmp/family1/646e129f33bc496bbec5919cb24f0e1e 2023-02-13 00:55:51,667 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021335d3eed7bc684ac2b78fcae374c291d2 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c2023021335d3eed7bc684ac2b78fcae374c291d2 2023-02-13 00:55:51,672 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/.tmp/family2/87fd172d68d24cf0ba98a63af3e3cc31 2023-02-13 00:55:51,675 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/.tmp/family1/a1ba8c22a68240d49802410b2775020e as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/a1ba8c22a68240d49802410b2775020e 2023-02-13 00:55:51,678 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/a1ba8c22a68240d49802410b2775020e, entries=4, sequenceid=9, filesize=5.4 K 2023-02-13 00:55:51,678 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/.tmp/family2/87fd172d68d24cf0ba98a63af3e3cc31 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/87fd172d68d24cf0ba98a63af3e3cc31 2023-02-13 00:55:51,681 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/87fd172d68d24cf0ba98a63af3e3cc31, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 00:55:51,682 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 914f10fbb08273a073f4cd678112ab73 in 848ms, sequenceid=9, compaction requested=false 2023-02-13 00:55:51,682 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 914f10fbb08273a073f4cd678112ab73: 2023-02-13 00:55:51,682 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. 2023-02-13 00:55:51,682 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 00:55:52,072 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021377ccb6b415a545f98bb30d78f22173cb to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023021377ccb6b415a545f98bb30d78f22173cb 2023-02-13 00:55:52,072 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302138d61fa0525cc43bd867257105168f043 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202302138d61fa0525cc43bd867257105168f043 2023-02-13 00:55:52,478 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/.tmp/family2/902c11a42b85484c934fabb6fb29693a 2023-02-13 00:55:52,478 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/.tmp/family2/1c42c61791fe4961a182c5433f688472 2023-02-13 00:55:52,481 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/.tmp/family1/646e129f33bc496bbec5919cb24f0e1e as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/646e129f33bc496bbec5919cb24f0e1e 2023-02-13 00:55:52,482 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/.tmp/family1/f379a72bb27448a19088382d0b086fc4 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/f379a72bb27448a19088382d0b086fc4 2023-02-13 00:55:52,485 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/646e129f33bc496bbec5919cb24f0e1e, entries=4, sequenceid=9, filesize=5.4 K 2023-02-13 00:55:52,485 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/f379a72bb27448a19088382d0b086fc4, entries=4, sequenceid=9, filesize=5.4 K 2023-02-13 00:55:52,486 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/.tmp/family2/1c42c61791fe4961a182c5433f688472 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/1c42c61791fe4961a182c5433f688472 2023-02-13 00:55:52,486 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/.tmp/family2/902c11a42b85484c934fabb6fb29693a as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/902c11a42b85484c934fabb6fb29693a 2023-02-13 00:55:52,489 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/1c42c61791fe4961a182c5433f688472, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 00:55:52,490 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/902c11a42b85484c934fabb6fb29693a, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 00:55:52,490 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for a55e824ba00709d09709c5889f9c7a42 in 1656ms, sequenceid=9, compaction requested=false 2023-02-13 00:55:52,490 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for a55e824ba00709d09709c5889f9c7a42: 2023-02-13 00:55:52,490 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. 2023-02-13 00:55:52,490 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 00:55:52,491 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 5bae73767db4da59da83135d4e2a56db in 1656ms, sequenceid=9, compaction requested=false 2023-02-13 00:55:52,491 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 5bae73767db4da59da83135d4e2a56db: 2023-02-13 00:55:52,491 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. 2023-02-13 00:55:52,491 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 00:55:52,491 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 00:55:52,491 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:52,491 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-13 00:55:52,491 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-13 00:55:52,493 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:52,493 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:52,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:52,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:52,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:52,494 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:52,494 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:52,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:52,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:52,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:52,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:52,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:52,495 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-13 00:55:52,495 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6d6a64fd[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:52,495 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-13 00:55:52,495 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:52,496 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:52,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:52,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:52,496 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-13 00:55:52,496 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-13 00:55:52,496 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:52,497 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:52,496 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:55:52,502 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-13 00:55:52,502 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:52,502 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:52,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:52,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:52,502 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-13 00:55:52,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:52,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:52,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:52,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:52,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:52,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:52,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:52,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:52,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:55:52,504 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-13 00:55:52,504 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-13 00:55:52,504 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:52,504 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:52,504 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-13 00:55:52,504 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:52,506 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:52,506 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:52,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:52,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:52,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:52,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:52,506 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:52,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:52,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:52,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:52,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:52,506 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:52,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:52,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:52,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:52,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:52,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:52,507 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:52,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:52,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:52,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:52,510 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:52,510 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:52,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:52,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:52,510 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:52,510 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:52,510 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-13 00:55:52,510 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:52,510 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 00:55:52,510 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 00:55:52,510 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:52,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:52,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:52,510 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:52,510 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-13 00:55:52,510 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:52,510 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:52,511 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:52,611 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:52,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:52,612 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:55:52,613 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:55:52,613 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:55:52,615 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:52,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-13 00:55:52,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-13 00:55:52,617 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-13 00:55:52,617 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:52,617 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-13 00:55:52,617 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-13 00:55:52,617 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:52,617 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 00:55:52,618 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:52,618 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:52,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:52,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:52,619 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:52,619 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:52,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 00:55:52,619 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:52,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:52,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 00:55:52,619 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-13 00:55:52,620 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-13 00:55:52,620 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-13 00:55:52,620 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-13 00:55:52,620 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:52,620 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-13 00:55:52,620 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. 2023-02-13 00:55:52,620 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. 2023-02-13 00:55:52,620 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. started... 2023-02-13 00:55:52,620 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 00:55:52,620 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:52,620 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. 2023-02-13 00:55:52,620 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. started... 2023-02-13 00:55:52,620 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. started... 2023-02-13 00:55:52,620 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 5bae73767db4da59da83135d4e2a56db 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 00:55:52,620 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 914f10fbb08273a073f4cd678112ab73 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 00:55:52,621 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing a55e824ba00709d09709c5889f9c7a42 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 00:55:52,640 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302130eb05a430d3b4f579b01b9575dfa4add to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202302130eb05a430d3b4f579b01b9575dfa4add 2023-02-13 00:55:53,039 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021373327bb3f9594a1cbf7e234dd3c73511 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023021373327bb3f9594a1cbf7e234dd3c73511 2023-02-13 00:55:53,040 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021369d1e4ed876c4668a0d797205040a831 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023021369d1e4ed876c4668a0d797205040a831 2023-02-13 00:55:53,046 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/.tmp/family1/4036cd080f6749c09282a435bb917b6a 2023-02-13 00:55:53,448 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/.tmp/family1/ffdfd5f259ab4444b3e6c0becd460162 2023-02-13 00:55:53,450 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/.tmp/family1/450bddf26486408589f26f0a327037c8 2023-02-13 00:55:53,460 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213b7b6720a1e144c7da5ae4088080a1fd9 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230213b7b6720a1e144c7da5ae4088080a1fd9 2023-02-13 00:55:53,574 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 00:55:53,861 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213b59a238ecf2a4bff96acf5209bb52880 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230213b59a238ecf2a4bff96acf5209bb52880 2023-02-13 00:55:53,861 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213de8bb64476fe4253968b9517fe9e697b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230213de8bb64476fe4253968b9517fe9e697b 2023-02-13 00:55:53,864 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/.tmp/family2/43eda8582d0240e69bb3a46677ca2c31 2023-02-13 00:55:53,868 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/.tmp/family1/4036cd080f6749c09282a435bb917b6a as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/4036cd080f6749c09282a435bb917b6a 2023-02-13 00:55:53,871 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/4036cd080f6749c09282a435bb917b6a, entries=4, sequenceid=13, filesize=5.4 K 2023-02-13 00:55:53,871 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/.tmp/family2/43eda8582d0240e69bb3a46677ca2c31 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/43eda8582d0240e69bb3a46677ca2c31 2023-02-13 00:55:53,874 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/43eda8582d0240e69bb3a46677ca2c31, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 00:55:53,875 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 914f10fbb08273a073f4cd678112ab73 in 1255ms, sequenceid=13, compaction requested=true 2023-02-13 00:55:53,875 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 914f10fbb08273a073f4cd678112ab73: 2023-02-13 00:55:53,875 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. 2023-02-13 00:55:53,875 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 00:55:54,266 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/.tmp/family2/b1081acea6e94ed0846e3a8b9796d70f 2023-02-13 00:55:54,266 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/.tmp/family2/e611d5031c1446a5acb151b004ef18a7 2023-02-13 00:55:54,270 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/.tmp/family1/450bddf26486408589f26f0a327037c8 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/450bddf26486408589f26f0a327037c8 2023-02-13 00:55:54,270 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/.tmp/family1/ffdfd5f259ab4444b3e6c0becd460162 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/ffdfd5f259ab4444b3e6c0becd460162 2023-02-13 00:55:54,273 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/450bddf26486408589f26f0a327037c8, entries=4, sequenceid=13, filesize=5.4 K 2023-02-13 00:55:54,274 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/ffdfd5f259ab4444b3e6c0becd460162, entries=4, sequenceid=13, filesize=5.4 K 2023-02-13 00:55:54,274 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/.tmp/family2/e611d5031c1446a5acb151b004ef18a7 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/e611d5031c1446a5acb151b004ef18a7 2023-02-13 00:55:54,274 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/.tmp/family2/b1081acea6e94ed0846e3a8b9796d70f as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/b1081acea6e94ed0846e3a8b9796d70f 2023-02-13 00:55:54,277 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/e611d5031c1446a5acb151b004ef18a7, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 00:55:54,278 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/b1081acea6e94ed0846e3a8b9796d70f, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 00:55:54,278 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 5bae73767db4da59da83135d4e2a56db in 1658ms, sequenceid=13, compaction requested=true 2023-02-13 00:55:54,278 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for a55e824ba00709d09709c5889f9c7a42 in 1658ms, sequenceid=13, compaction requested=true 2023-02-13 00:55:54,278 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 5bae73767db4da59da83135d4e2a56db: 2023-02-13 00:55:54,278 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for a55e824ba00709d09709c5889f9c7a42: 2023-02-13 00:55:54,278 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. 2023-02-13 00:55:54,278 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. 2023-02-13 00:55:54,278 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 00:55:54,278 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 00:55:54,278 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 00:55:54,278 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:54,279 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-13 00:55:54,279 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-13 00:55:54,281 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:54,281 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:54,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:54,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:54,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:54,281 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:54,281 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:54,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:54,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:54,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:54,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:54,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:54,283 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-13 00:55:54,283 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24d4a010[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:54,283 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-13 00:55:54,283 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:54,284 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:54,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:54,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:54,284 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-13 00:55:54,284 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-13 00:55:54,284 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:55:54,284 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:54,284 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:54,285 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-13 00:55:54,285 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:54,285 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:54,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:54,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:54,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:54,285 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-13 00:55:54,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:54,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:54,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:54,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:54,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:54,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:54,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:54,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:55:54,287 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-13 00:55:54,287 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-13 00:55:54,287 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:54,287 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:54,287 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-13 00:55:54,287 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:54,289 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:54,289 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:54,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:54,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:54,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:54,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:54,289 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:54,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:54,289 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:54,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:54,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:54,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:54,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:54,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:54,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:54,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:54,290 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:54,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:54,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:54,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:54,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:54,294 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:54,294 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 00:55:54,294 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 00:55:54,294 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:54,294 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:54,294 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-13 00:55:54,294 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:54,294 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:54,294 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:54,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:54,294 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-13 00:55:54,294 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:54,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:54,294 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:54,294 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-13 00:55:54,294 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:54,294 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:54,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:54,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:54,395 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:54,395 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:54,396 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:55:54,396 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:55:54,397 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:55:54,399 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:54,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-13 00:55:54,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-13 00:55:54,400 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-13 00:55:54,400 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:54,400 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-13 00:55:54,400 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-13 00:55:54,401 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:54,401 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 00:55:54,402 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:54,402 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:54,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:54,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:54,403 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:54,403 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:54,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 00:55:54,403 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:54,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:54,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 00:55:54,403 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-13 00:55:54,403 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-13 00:55:54,403 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-13 00:55:54,403 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-13 00:55:54,404 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:54,404 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-13 00:55:54,404 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. 2023-02-13 00:55:54,404 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 00:55:54,404 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:54,404 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. 2023-02-13 00:55:54,404 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. started... 2023-02-13 00:55:54,404 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. started... 2023-02-13 00:55:54,404 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. 2023-02-13 00:55:54,404 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 5bae73767db4da59da83135d4e2a56db 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 00:55:54,404 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 914f10fbb08273a073f4cd678112ab73 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 00:55:54,404 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. started... 2023-02-13 00:55:54,404 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing a55e824ba00709d09709c5889f9c7a42 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 00:55:54,415 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213a94c03f1a92743e4b597f85d354c8c1e to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230213a94c03f1a92743e4b597f85d354c8c1e 2023-02-13 00:55:54,415 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213228510528fe94db69b0eea22c9b09f1f to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230213228510528fe94db69b0eea22c9b09f1f 2023-02-13 00:55:54,815 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302133f4610df8fdb4c44ae616695f331db2c to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202302133f4610df8fdb4c44ae616695f331db2c 2023-02-13 00:55:54,819 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/.tmp/family1/e2085eecbca7434aa9111b10672d64cb 2023-02-13 00:55:54,820 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/.tmp/family1/c363b3ae37ec410eb07c914c9e95ae86 2023-02-13 00:55:55,220 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/.tmp/family1/672269ae4a3d43e68db3c549adf59de2 2023-02-13 00:55:55,231 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213e38e8aceaf834cc6a06edef6323a55a4 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230213e38e8aceaf834cc6a06edef6323a55a4 2023-02-13 00:55:55,231 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213a61f4d7bf3e84c2c86c35a88246c97a3 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230213a61f4d7bf3e84c2c86c35a88246c97a3 2023-02-13 00:55:55,232 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213d8b850b94e824ab7a52b0e28dc5ab93f to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230213d8b850b94e824ab7a52b0e28dc5ab93f 2023-02-13 00:55:55,636 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/.tmp/family2/f7410868bc914750b8c2006bf1e3e243 2023-02-13 00:55:55,636 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/.tmp/family2/95798a2f2831432582bd36084d17c46b 2023-02-13 00:55:55,636 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/.tmp/family2/b61a93aca649447fa177c84278b1474c 2023-02-13 00:55:55,640 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/.tmp/family1/672269ae4a3d43e68db3c549adf59de2 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/672269ae4a3d43e68db3c549adf59de2 2023-02-13 00:55:55,640 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/.tmp/family1/e2085eecbca7434aa9111b10672d64cb as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/e2085eecbca7434aa9111b10672d64cb 2023-02-13 00:55:55,640 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/.tmp/family1/c363b3ae37ec410eb07c914c9e95ae86 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/c363b3ae37ec410eb07c914c9e95ae86 2023-02-13 00:55:55,643 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/672269ae4a3d43e68db3c549adf59de2, entries=4, sequenceid=17, filesize=5.4 K 2023-02-13 00:55:55,643 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/c363b3ae37ec410eb07c914c9e95ae86, entries=4, sequenceid=17, filesize=5.4 K 2023-02-13 00:55:55,644 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/e2085eecbca7434aa9111b10672d64cb, entries=4, sequenceid=17, filesize=5.4 K 2023-02-13 00:55:55,644 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/.tmp/family2/b61a93aca649447fa177c84278b1474c as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/b61a93aca649447fa177c84278b1474c 2023-02-13 00:55:55,644 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/.tmp/family2/f7410868bc914750b8c2006bf1e3e243 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/f7410868bc914750b8c2006bf1e3e243 2023-02-13 00:55:55,644 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/.tmp/family2/95798a2f2831432582bd36084d17c46b as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/95798a2f2831432582bd36084d17c46b 2023-02-13 00:55:55,648 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/f7410868bc914750b8c2006bf1e3e243, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 00:55:55,648 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/95798a2f2831432582bd36084d17c46b, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 00:55:55,648 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/b61a93aca649447fa177c84278b1474c, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 00:55:55,649 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 5bae73767db4da59da83135d4e2a56db in 1245ms, sequenceid=17, compaction requested=true 2023-02-13 00:55:55,649 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for a55e824ba00709d09709c5889f9c7a42 in 1245ms, sequenceid=17, compaction requested=true 2023-02-13 00:55:55,649 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 5bae73767db4da59da83135d4e2a56db: 2023-02-13 00:55:55,649 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for a55e824ba00709d09709c5889f9c7a42: 2023-02-13 00:55:55,649 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 914f10fbb08273a073f4cd678112ab73 in 1245ms, sequenceid=17, compaction requested=true 2023-02-13 00:55:55,649 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. 2023-02-13 00:55:55,649 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 914f10fbb08273a073f4cd678112ab73: 2023-02-13 00:55:55,649 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 00:55:55,649 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. 2023-02-13 00:55:55,649 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. 2023-02-13 00:55:55,649 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 00:55:55,649 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 00:55:55,649 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 00:55:55,649 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:55,649 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-13 00:55:55,649 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-13 00:55:55,652 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:55,652 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:55,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:55,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:55,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:55,653 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:55,653 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:55,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:55,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:55,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:55,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:55,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:55,654 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-13 00:55:55,654 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53e92b40[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:55,654 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-13 00:55:55,654 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:55,657 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:55,657 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:55,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:55,657 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-13 00:55:55,657 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-13 00:55:55,657 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:55:55,657 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:55,657 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:55,659 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-13 00:55:55,659 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:55,659 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:55,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:55,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:55,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:55,659 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-13 00:55:55,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:55,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:55,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:55,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:55,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:55,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:55,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:55,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:55:55,662 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-13 00:55:55,662 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-13 00:55:55,662 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:55,662 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:55,662 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-13 00:55:55,662 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:55,664 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:55,664 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:55,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:55,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:55,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:55,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:55,664 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:55,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:55,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:55,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:55,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:55,665 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:55,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:55,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:55,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:55,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:55,666 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:55,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:55,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:55,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:55,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:55,668 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:55,668 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:55,668 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:55,668 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:55,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:55,668 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-13 00:55:55,669 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:55,668 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-13 00:55:55,668 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 00:55:55,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:55,669 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 00:55:55,669 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:55,669 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:55,669 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:55,669 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:55,669 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-13 00:55:55,669 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:55,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:55,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:55,769 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:55,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:55,780 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 00:55:55,789 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 00:55:55,796 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 00:55:55,797 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 00:55:55,821 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:55,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-13 00:55:55,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-13 00:55:55,824 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-13 00:55:55,824 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:55,824 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-13 00:55:55,824 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-13 00:55:55,824 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:55,824 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 00:55:55,826 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:55,826 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:55,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:55,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:55,826 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:55,826 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:55,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 00:55:55,826 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:55,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:55,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 00:55:55,827 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-13 00:55:55,827 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-13 00:55:55,827 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-13 00:55:55,827 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-13 00:55:55,827 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:55,827 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-13 00:55:55,827 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. 2023-02-13 00:55:55,827 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. started... 2023-02-13 00:55:55,827 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. 2023-02-13 00:55:55,827 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. started... 2023-02-13 00:55:55,827 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 914f10fbb08273a073f4cd678112ab73 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 00:55:55,827 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 5bae73767db4da59da83135d4e2a56db 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 00:55:55,828 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 00:55:55,828 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:55,828 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. 2023-02-13 00:55:55,828 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. started... 2023-02-13 00:55:55,828 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing a55e824ba00709d09709c5889f9c7a42 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 00:55:55,862 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/.tmp/family1/f7061b816aa64fcd82ce50cce7347c8a 2023-02-13 00:55:55,865 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f7061b816aa64fcd82ce50cce7347c8a 2023-02-13 00:55:56,656 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/.tmp/family1/4e1d100546704479b42f41454d02e50a 2023-02-13 00:55:56,656 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/.tmp/family1/0a70fa9f31284093bb9b402845cefa9c 2023-02-13 00:55:56,660 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4e1d100546704479b42f41454d02e50a 2023-02-13 00:55:56,660 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0a70fa9f31284093bb9b402845cefa9c 2023-02-13 00:55:56,679 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/.tmp/family2/e762d0689616462695e589a9fdd813f3 2023-02-13 00:55:56,682 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e762d0689616462695e589a9fdd813f3 2023-02-13 00:55:56,683 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/.tmp/family1/f7061b816aa64fcd82ce50cce7347c8a as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/f7061b816aa64fcd82ce50cce7347c8a 2023-02-13 00:55:56,686 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f7061b816aa64fcd82ce50cce7347c8a 2023-02-13 00:55:56,687 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/f7061b816aa64fcd82ce50cce7347c8a, entries=3, sequenceid=23, filesize=5.0 K 2023-02-13 00:55:56,687 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/.tmp/family2/e762d0689616462695e589a9fdd813f3 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/e762d0689616462695e589a9fdd813f3 2023-02-13 00:55:56,690 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e762d0689616462695e589a9fdd813f3 2023-02-13 00:55:56,691 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/e762d0689616462695e589a9fdd813f3, entries=1, sequenceid=23, filesize=5.0 K 2023-02-13 00:55:56,691 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for a55e824ba00709d09709c5889f9c7a42 in 863ms, sequenceid=23, compaction requested=true 2023-02-13 00:55:56,691 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for a55e824ba00709d09709c5889f9c7a42: 2023-02-13 00:55:56,691 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. 2023-02-13 00:55:56,691 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 00:55:57,478 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/.tmp/family2/6b10857664124888b52d38234e2669e7 2023-02-13 00:55:57,479 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/.tmp/family2/2e51e62de0234457a6b2fad2bf6e1324 2023-02-13 00:55:57,482 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2e51e62de0234457a6b2fad2bf6e1324 2023-02-13 00:55:57,482 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6b10857664124888b52d38234e2669e7 2023-02-13 00:55:57,483 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/.tmp/family1/4e1d100546704479b42f41454d02e50a as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/4e1d100546704479b42f41454d02e50a 2023-02-13 00:55:57,483 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/.tmp/family1/0a70fa9f31284093bb9b402845cefa9c as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/0a70fa9f31284093bb9b402845cefa9c 2023-02-13 00:55:57,486 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4e1d100546704479b42f41454d02e50a 2023-02-13 00:55:57,486 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/4e1d100546704479b42f41454d02e50a, entries=3, sequenceid=23, filesize=5.0 K 2023-02-13 00:55:57,486 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0a70fa9f31284093bb9b402845cefa9c 2023-02-13 00:55:57,486 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/0a70fa9f31284093bb9b402845cefa9c, entries=3, sequenceid=23, filesize=5.0 K 2023-02-13 00:55:57,487 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/.tmp/family2/2e51e62de0234457a6b2fad2bf6e1324 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/2e51e62de0234457a6b2fad2bf6e1324 2023-02-13 00:55:57,487 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/.tmp/family2/6b10857664124888b52d38234e2669e7 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/6b10857664124888b52d38234e2669e7 2023-02-13 00:55:57,490 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2e51e62de0234457a6b2fad2bf6e1324 2023-02-13 00:55:57,490 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/2e51e62de0234457a6b2fad2bf6e1324, entries=1, sequenceid=23, filesize=5.0 K 2023-02-13 00:55:57,490 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6b10857664124888b52d38234e2669e7 2023-02-13 00:55:57,490 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/6b10857664124888b52d38234e2669e7, entries=1, sequenceid=23, filesize=5.0 K 2023-02-13 00:55:57,491 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 914f10fbb08273a073f4cd678112ab73 in 1664ms, sequenceid=23, compaction requested=true 2023-02-13 00:55:57,491 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 5bae73767db4da59da83135d4e2a56db in 1664ms, sequenceid=23, compaction requested=true 2023-02-13 00:55:57,491 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 914f10fbb08273a073f4cd678112ab73: 2023-02-13 00:55:57,491 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 5bae73767db4da59da83135d4e2a56db: 2023-02-13 00:55:57,491 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. 2023-02-13 00:55:57,491 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. 2023-02-13 00:55:57,491 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 00:55:57,491 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 00:55:57,491 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 00:55:57,491 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:57,491 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-13 00:55:57,491 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-13 00:55:57,493 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:57,493 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:57,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:57,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:57,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:57,493 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:57,494 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:57,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:57,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:57,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:57,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:57,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:57,495 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-13 00:55:57,495 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@714d8818[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:57,495 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-13 00:55:57,495 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:57,496 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:57,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:57,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:57,496 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-13 00:55:57,496 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-13 00:55:57,496 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:55:57,496 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:57,496 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:57,497 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:57,497 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-13 00:55:57,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:57,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:57,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:57,497 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:57,497 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-13 00:55:57,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:57,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:57,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:57,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:57,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:57,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:57,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:57,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:55:57,499 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-13 00:55:57,499 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-13 00:55:57,499 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:57,499 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:57,499 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-13 00:55:57,499 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:57,505 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:57,505 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:57,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:57,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:57,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:57,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:57,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:57,505 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:57,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:57,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:57,505 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:57,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:57,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:57,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:57,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:57,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:57,506 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:57,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:57,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:57,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 00:55:57,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:57,508 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:57,508 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:57,508 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 00:55:57,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:57,508 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-13 00:55:57,508 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-13 00:55:57,508 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:57,508 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:57,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:57,508 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:57,508 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 00:55:57,509 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:55:57,509 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:57,509 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 00:55:57,509 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 00:55:57,509 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-13 00:55:57,509 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:57,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:57,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:57,609 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:57,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:57,610 DEBUG [Listener at localhost/37869] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 00:55:57,610 DEBUG [Listener at localhost/37869] regionserver.HStore(1912): 914f10fbb08273a073f4cd678112ab73/family1 is initiating major compaction (all files) 2023-02-13 00:55:57,610 INFO [Listener at localhost/37869] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 00:55:57,610 INFO [Listener at localhost/37869] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:55:57,610 INFO [Listener at localhost/37869] regionserver.HRegion(2258): Starting compaction of 914f10fbb08273a073f4cd678112ab73/family1 in ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. 2023-02-13 00:55:57,611 INFO [Listener at localhost/37869] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/70ec1e114290430287f28713178962f6, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/a1ba8c22a68240d49802410b2775020e, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/4036cd080f6749c09282a435bb917b6a, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/672269ae4a3d43e68db3c549adf59de2, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/4e1d100546704479b42f41454d02e50a] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/.tmp, totalSize=26.6 K 2023-02-13 00:55:57,611 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 70ec1e114290430287f28713178962f6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676249749039 2023-02-13 00:55:57,611 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting a1ba8c22a68240d49802410b2775020e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676249750827 2023-02-13 00:55:57,612 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 4036cd080f6749c09282a435bb917b6a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676249752613 2023-02-13 00:55:57,612 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 672269ae4a3d43e68db3c549adf59de2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676249754397 2023-02-13 00:55:57,612 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 4e1d100546704479b42f41454d02e50a, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 00:55:57,620 INFO [Listener at localhost/37869] throttle.PressureAwareThroughputController(145): 914f10fbb08273a073f4cd678112ab73#family1#compaction#149 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:55:58,033 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202302134f16ced26407438fbebbd2b9834ea796_del 2023-02-13 00:55:58,033 INFO [Listener at localhost/37869] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302134f16ced26407438fbebbd2b9834ea796_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202302134f16ced26407438fbebbd2b9834ea796_del 2023-02-13 00:55:58,845 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5cf2846f57e24cff94d0710e8e6a27ae 2023-02-13 00:55:58,846 DEBUG [Listener at localhost/37869] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/.tmp/family1/5cf2846f57e24cff94d0710e8e6a27ae as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/5cf2846f57e24cff94d0710e8e6a27ae 2023-02-13 00:55:58,850 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5cf2846f57e24cff94d0710e8e6a27ae 2023-02-13 00:55:58,851 INFO [Listener at localhost/37869] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 914f10fbb08273a073f4cd678112ab73/family1 of 914f10fbb08273a073f4cd678112ab73 into 5cf2846f57e24cff94d0710e8e6a27ae(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 00:55:58,851 DEBUG [Listener at localhost/37869] regionserver.HRegion(2288): Compaction status journal for 914f10fbb08273a073f4cd678112ab73: 2023-02-13 00:55:58,851 DEBUG [Listener at localhost/37869] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 00:55:58,851 DEBUG [Listener at localhost/37869] regionserver.HStore(1912): 914f10fbb08273a073f4cd678112ab73/family2 is initiating major compaction (all files) 2023-02-13 00:55:58,851 INFO [Listener at localhost/37869] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 00:55:58,851 INFO [Listener at localhost/37869] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:55:58,852 INFO [Listener at localhost/37869] regionserver.HRegion(2258): Starting compaction of 914f10fbb08273a073f4cd678112ab73/family2 in ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. 2023-02-13 00:55:58,852 INFO [Listener at localhost/37869] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/53bf28f082504c7c8d8c2b1aa5f9879d, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/87fd172d68d24cf0ba98a63af3e3cc31, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/43eda8582d0240e69bb3a46677ca2c31, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/b61a93aca649447fa177c84278b1474c, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/2e51e62de0234457a6b2fad2bf6e1324] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/.tmp, totalSize=25.3 K 2023-02-13 00:55:58,853 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 53bf28f082504c7c8d8c2b1aa5f9879d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676249749039 2023-02-13 00:55:58,853 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 87fd172d68d24cf0ba98a63af3e3cc31, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676249750827 2023-02-13 00:55:58,853 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 43eda8582d0240e69bb3a46677ca2c31, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676249752613 2023-02-13 00:55:58,854 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting b61a93aca649447fa177c84278b1474c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676249754397 2023-02-13 00:55:58,854 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 2e51e62de0234457a6b2fad2bf6e1324, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 00:55:58,862 INFO [Listener at localhost/37869] throttle.PressureAwareThroughputController(145): 914f10fbb08273a073f4cd678112ab73#family2#compaction#150 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:55:58,936 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/70ec1e114290430287f28713178962f6, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/a1ba8c22a68240d49802410b2775020e, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/4036cd080f6749c09282a435bb917b6a, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/672269ae4a3d43e68db3c549adf59de2, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/4e1d100546704479b42f41454d02e50a] to archive 2023-02-13 00:55:58,937 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:58,939 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/70ec1e114290430287f28713178962f6 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/70ec1e114290430287f28713178962f6 2023-02-13 00:55:58,939 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/a1ba8c22a68240d49802410b2775020e to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/a1ba8c22a68240d49802410b2775020e 2023-02-13 00:55:58,940 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/4036cd080f6749c09282a435bb917b6a to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/4036cd080f6749c09282a435bb917b6a 2023-02-13 00:55:58,941 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/672269ae4a3d43e68db3c549adf59de2 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/672269ae4a3d43e68db3c549adf59de2 2023-02-13 00:55:58,942 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/4e1d100546704479b42f41454d02e50a to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/4e1d100546704479b42f41454d02e50a 2023-02-13 00:55:59,270 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230213d0aab7c15e2f48ac9ede7258bfbaac92_del 2023-02-13 00:55:59,270 INFO [Listener at localhost/37869] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213d0aab7c15e2f48ac9ede7258bfbaac92_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230213d0aab7c15e2f48ac9ede7258bfbaac92_del 2023-02-13 00:56:00,083 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 42f74a45d2f94134af51e20723f3de98 2023-02-13 00:56:00,083 DEBUG [Listener at localhost/37869] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/.tmp/family2/42f74a45d2f94134af51e20723f3de98 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/42f74a45d2f94134af51e20723f3de98 2023-02-13 00:56:00,087 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 42f74a45d2f94134af51e20723f3de98 2023-02-13 00:56:00,087 INFO [Listener at localhost/37869] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 914f10fbb08273a073f4cd678112ab73/family2 of 914f10fbb08273a073f4cd678112ab73 into 42f74a45d2f94134af51e20723f3de98(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 00:56:00,088 DEBUG [Listener at localhost/37869] regionserver.HRegion(2288): Compaction status journal for 914f10fbb08273a073f4cd678112ab73: 2023-02-13 00:56:00,088 DEBUG [Listener at localhost/37869] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 00:56:00,088 DEBUG [Listener at localhost/37869] regionserver.HStore(1912): 5bae73767db4da59da83135d4e2a56db/family1 is initiating major compaction (all files) 2023-02-13 00:56:00,088 INFO [Listener at localhost/37869] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 00:56:00,088 INFO [Listener at localhost/37869] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:56:00,088 INFO [Listener at localhost/37869] regionserver.HRegion(2258): Starting compaction of 5bae73767db4da59da83135d4e2a56db/family1 in ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. 2023-02-13 00:56:00,088 INFO [Listener at localhost/37869] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/d1e6cd4f51f5432086b939cb3083450d, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/f379a72bb27448a19088382d0b086fc4, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/ffdfd5f259ab4444b3e6c0becd460162, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/c363b3ae37ec410eb07c914c9e95ae86, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/0a70fa9f31284093bb9b402845cefa9c] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/.tmp, totalSize=26.6 K 2023-02-13 00:56:00,088 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting d1e6cd4f51f5432086b939cb3083450d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676249749039 2023-02-13 00:56:00,089 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting f379a72bb27448a19088382d0b086fc4, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676249750826 2023-02-13 00:56:00,089 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting ffdfd5f259ab4444b3e6c0becd460162, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676249752612 2023-02-13 00:56:00,089 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting c363b3ae37ec410eb07c914c9e95ae86, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676249754396 2023-02-13 00:56:00,090 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 0a70fa9f31284093bb9b402845cefa9c, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 00:56:00,098 INFO [Listener at localhost/37869] throttle.PressureAwareThroughputController(145): 5bae73767db4da59da83135d4e2a56db#family1#compaction#151 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:56:00,136 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/53bf28f082504c7c8d8c2b1aa5f9879d, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/87fd172d68d24cf0ba98a63af3e3cc31, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/43eda8582d0240e69bb3a46677ca2c31, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/b61a93aca649447fa177c84278b1474c, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/2e51e62de0234457a6b2fad2bf6e1324] to archive 2023-02-13 00:56:00,137 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:00,138 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/53bf28f082504c7c8d8c2b1aa5f9879d to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/53bf28f082504c7c8d8c2b1aa5f9879d 2023-02-13 00:56:00,139 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/87fd172d68d24cf0ba98a63af3e3cc31 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/87fd172d68d24cf0ba98a63af3e3cc31 2023-02-13 00:56:00,140 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/43eda8582d0240e69bb3a46677ca2c31 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/43eda8582d0240e69bb3a46677ca2c31 2023-02-13 00:56:00,140 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/b61a93aca649447fa177c84278b1474c to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/b61a93aca649447fa177c84278b1474c 2023-02-13 00:56:00,141 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/2e51e62de0234457a6b2fad2bf6e1324 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family2/2e51e62de0234457a6b2fad2bf6e1324 2023-02-13 00:56:00,505 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302132d85dc70c6a84b44b58796d8aaf700a9_del 2023-02-13 00:56:00,505 INFO [Listener at localhost/37869] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302132d85dc70c6a84b44b58796d8aaf700a9_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202302132d85dc70c6a84b44b58796d8aaf700a9_del 2023-02-13 00:56:01,317 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2e744b944c6b4a30800a120214e48dd9 2023-02-13 00:56:01,318 DEBUG [Listener at localhost/37869] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/.tmp/family1/2e744b944c6b4a30800a120214e48dd9 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/2e744b944c6b4a30800a120214e48dd9 2023-02-13 00:56:01,322 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2e744b944c6b4a30800a120214e48dd9 2023-02-13 00:56:01,323 INFO [Listener at localhost/37869] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 5bae73767db4da59da83135d4e2a56db/family1 of 5bae73767db4da59da83135d4e2a56db into 2e744b944c6b4a30800a120214e48dd9(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 00:56:01,323 DEBUG [Listener at localhost/37869] regionserver.HRegion(2288): Compaction status journal for 5bae73767db4da59da83135d4e2a56db: 2023-02-13 00:56:01,323 DEBUG [Listener at localhost/37869] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 00:56:01,323 DEBUG [Listener at localhost/37869] regionserver.HStore(1912): 5bae73767db4da59da83135d4e2a56db/family2 is initiating major compaction (all files) 2023-02-13 00:56:01,323 INFO [Listener at localhost/37869] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 00:56:01,323 INFO [Listener at localhost/37869] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:56:01,323 INFO [Listener at localhost/37869] regionserver.HRegion(2258): Starting compaction of 5bae73767db4da59da83135d4e2a56db/family2 in ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. 2023-02-13 00:56:01,323 INFO [Listener at localhost/37869] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/95dec832c3054bcabcd0399055e2efc3, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/902c11a42b85484c934fabb6fb29693a, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/b1081acea6e94ed0846e3a8b9796d70f, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/f7410868bc914750b8c2006bf1e3e243, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/6b10857664124888b52d38234e2669e7] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/.tmp, totalSize=25.3 K 2023-02-13 00:56:01,324 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 95dec832c3054bcabcd0399055e2efc3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676249749039 2023-02-13 00:56:01,324 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 902c11a42b85484c934fabb6fb29693a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676249750826 2023-02-13 00:56:01,324 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting b1081acea6e94ed0846e3a8b9796d70f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676249752612 2023-02-13 00:56:01,324 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting f7410868bc914750b8c2006bf1e3e243, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676249754396 2023-02-13 00:56:01,325 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 6b10857664124888b52d38234e2669e7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 00:56:01,332 INFO [Listener at localhost/37869] throttle.PressureAwareThroughputController(145): 5bae73767db4da59da83135d4e2a56db#family2#compaction#152 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:56:01,336 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/d1e6cd4f51f5432086b939cb3083450d, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/f379a72bb27448a19088382d0b086fc4, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/ffdfd5f259ab4444b3e6c0becd460162, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/c363b3ae37ec410eb07c914c9e95ae86, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/0a70fa9f31284093bb9b402845cefa9c] to archive 2023-02-13 00:56:01,336 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:01,337 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/d1e6cd4f51f5432086b939cb3083450d to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/d1e6cd4f51f5432086b939cb3083450d 2023-02-13 00:56:01,338 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/f379a72bb27448a19088382d0b086fc4 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/f379a72bb27448a19088382d0b086fc4 2023-02-13 00:56:01,339 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/ffdfd5f259ab4444b3e6c0becd460162 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/ffdfd5f259ab4444b3e6c0becd460162 2023-02-13 00:56:01,340 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/c363b3ae37ec410eb07c914c9e95ae86 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/c363b3ae37ec410eb07c914c9e95ae86 2023-02-13 00:56:01,341 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/0a70fa9f31284093bb9b402845cefa9c to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/0a70fa9f31284093bb9b402845cefa9c 2023-02-13 00:56:01,739 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302136fdbb93ed0854606ba26b8f2a271054a_del 2023-02-13 00:56:01,739 INFO [Listener at localhost/37869] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302136fdbb93ed0854606ba26b8f2a271054a_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202302136fdbb93ed0854606ba26b8f2a271054a_del 2023-02-13 00:56:02,553 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5f65c387ec4845e7a569f66294840932 2023-02-13 00:56:02,553 DEBUG [Listener at localhost/37869] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/.tmp/family2/5f65c387ec4845e7a569f66294840932 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/5f65c387ec4845e7a569f66294840932 2023-02-13 00:56:02,557 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5f65c387ec4845e7a569f66294840932 2023-02-13 00:56:02,557 INFO [Listener at localhost/37869] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 5bae73767db4da59da83135d4e2a56db/family2 of 5bae73767db4da59da83135d4e2a56db into 5f65c387ec4845e7a569f66294840932(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 00:56:02,558 DEBUG [Listener at localhost/37869] regionserver.HRegion(2288): Compaction status journal for 5bae73767db4da59da83135d4e2a56db: 2023-02-13 00:56:02,558 DEBUG [Listener at localhost/37869] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 00:56:02,558 DEBUG [Listener at localhost/37869] regionserver.HStore(1912): a55e824ba00709d09709c5889f9c7a42/family1 is initiating major compaction (all files) 2023-02-13 00:56:02,558 INFO [Listener at localhost/37869] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 00:56:02,558 INFO [Listener at localhost/37869] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:56:02,558 INFO [Listener at localhost/37869] regionserver.HRegion(2258): Starting compaction of a55e824ba00709d09709c5889f9c7a42/family1 in ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. 2023-02-13 00:56:02,558 INFO [Listener at localhost/37869] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/ef7baba1791d4087a97eaa0aa6a5ec6d, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/646e129f33bc496bbec5919cb24f0e1e, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/450bddf26486408589f26f0a327037c8, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/e2085eecbca7434aa9111b10672d64cb, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/f7061b816aa64fcd82ce50cce7347c8a] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/.tmp, totalSize=26.6 K 2023-02-13 00:56:02,558 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting ef7baba1791d4087a97eaa0aa6a5ec6d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676249749039 2023-02-13 00:56:02,559 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 646e129f33bc496bbec5919cb24f0e1e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676249750826 2023-02-13 00:56:02,559 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 450bddf26486408589f26f0a327037c8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676249752613 2023-02-13 00:56:02,559 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting e2085eecbca7434aa9111b10672d64cb, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676249754396 2023-02-13 00:56:02,560 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting f7061b816aa64fcd82ce50cce7347c8a, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 00:56:02,567 INFO [Listener at localhost/37869] throttle.PressureAwareThroughputController(145): a55e824ba00709d09709c5889f9c7a42#family1#compaction#153 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:56:02,638 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/95dec832c3054bcabcd0399055e2efc3, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/902c11a42b85484c934fabb6fb29693a, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/b1081acea6e94ed0846e3a8b9796d70f, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/f7410868bc914750b8c2006bf1e3e243, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/6b10857664124888b52d38234e2669e7] to archive 2023-02-13 00:56:02,638 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:02,640 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/95dec832c3054bcabcd0399055e2efc3 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/95dec832c3054bcabcd0399055e2efc3 2023-02-13 00:56:02,641 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/902c11a42b85484c934fabb6fb29693a to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/902c11a42b85484c934fabb6fb29693a 2023-02-13 00:56:02,642 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/b1081acea6e94ed0846e3a8b9796d70f to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/b1081acea6e94ed0846e3a8b9796d70f 2023-02-13 00:56:02,643 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/f7410868bc914750b8c2006bf1e3e243 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/f7410868bc914750b8c2006bf1e3e243 2023-02-13 00:56:02,644 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/6b10857664124888b52d38234e2669e7 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family2/6b10857664124888b52d38234e2669e7 2023-02-13 00:56:02,977 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230213f924c6c4cbfd42939034ef7cb43ead07_del 2023-02-13 00:56:02,977 INFO [Listener at localhost/37869] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213f924c6c4cbfd42939034ef7cb43ead07_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230213f924c6c4cbfd42939034ef7cb43ead07_del 2023-02-13 00:56:03,789 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6e5dd457bce145bab441acbabe0d8166 2023-02-13 00:56:03,790 DEBUG [Listener at localhost/37869] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/.tmp/family1/6e5dd457bce145bab441acbabe0d8166 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/6e5dd457bce145bab441acbabe0d8166 2023-02-13 00:56:03,793 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6e5dd457bce145bab441acbabe0d8166 2023-02-13 00:56:03,794 INFO [Listener at localhost/37869] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a55e824ba00709d09709c5889f9c7a42/family1 of a55e824ba00709d09709c5889f9c7a42 into 6e5dd457bce145bab441acbabe0d8166(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 00:56:03,794 DEBUG [Listener at localhost/37869] regionserver.HRegion(2288): Compaction status journal for a55e824ba00709d09709c5889f9c7a42: 2023-02-13 00:56:03,794 DEBUG [Listener at localhost/37869] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 00:56:03,794 DEBUG [Listener at localhost/37869] regionserver.HStore(1912): a55e824ba00709d09709c5889f9c7a42/family2 is initiating major compaction (all files) 2023-02-13 00:56:03,794 INFO [Listener at localhost/37869] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 00:56:03,794 INFO [Listener at localhost/37869] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:56:03,795 INFO [Listener at localhost/37869] regionserver.HRegion(2258): Starting compaction of a55e824ba00709d09709c5889f9c7a42/family2 in ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. 2023-02-13 00:56:03,795 INFO [Listener at localhost/37869] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/1b93672aad624b66b5769229e62e5264, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/1c42c61791fe4961a182c5433f688472, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/e611d5031c1446a5acb151b004ef18a7, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/95798a2f2831432582bd36084d17c46b, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/e762d0689616462695e589a9fdd813f3] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/.tmp, totalSize=25.3 K 2023-02-13 00:56:03,795 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 1b93672aad624b66b5769229e62e5264, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676249749039 2023-02-13 00:56:03,796 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 1c42c61791fe4961a182c5433f688472, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676249750826 2023-02-13 00:56:03,796 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting e611d5031c1446a5acb151b004ef18a7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676249752613 2023-02-13 00:56:03,796 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 95798a2f2831432582bd36084d17c46b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676249754396 2023-02-13 00:56:03,796 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting e762d0689616462695e589a9fdd813f3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 00:56:03,804 INFO [Listener at localhost/37869] throttle.PressureAwareThroughputController(145): a55e824ba00709d09709c5889f9c7a42#family2#compaction#154 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:56:03,836 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/ef7baba1791d4087a97eaa0aa6a5ec6d, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/646e129f33bc496bbec5919cb24f0e1e, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/450bddf26486408589f26f0a327037c8, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/e2085eecbca7434aa9111b10672d64cb, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/f7061b816aa64fcd82ce50cce7347c8a] to archive 2023-02-13 00:56:03,836 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:03,838 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/ef7baba1791d4087a97eaa0aa6a5ec6d to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/ef7baba1791d4087a97eaa0aa6a5ec6d 2023-02-13 00:56:03,839 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/646e129f33bc496bbec5919cb24f0e1e to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/646e129f33bc496bbec5919cb24f0e1e 2023-02-13 00:56:03,840 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/450bddf26486408589f26f0a327037c8 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/450bddf26486408589f26f0a327037c8 2023-02-13 00:56:03,840 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/e2085eecbca7434aa9111b10672d64cb to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/e2085eecbca7434aa9111b10672d64cb 2023-02-13 00:56:03,841 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/f7061b816aa64fcd82ce50cce7347c8a to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/f7061b816aa64fcd82ce50cce7347c8a 2023-02-13 00:56:04,212 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230213db09e64ceada48219f5aa0a22d4dcd5d_del 2023-02-13 00:56:04,212 INFO [Listener at localhost/37869] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213db09e64ceada48219f5aa0a22d4dcd5d_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230213db09e64ceada48219f5aa0a22d4dcd5d_del 2023-02-13 00:56:05,026 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c2ca55c2c8dd40d4a037d1dd3a8c59e3 2023-02-13 00:56:05,027 DEBUG [Listener at localhost/37869] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/.tmp/family2/c2ca55c2c8dd40d4a037d1dd3a8c59e3 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/c2ca55c2c8dd40d4a037d1dd3a8c59e3 2023-02-13 00:56:05,030 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c2ca55c2c8dd40d4a037d1dd3a8c59e3 2023-02-13 00:56:05,031 INFO [Listener at localhost/37869] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a55e824ba00709d09709c5889f9c7a42/family2 of a55e824ba00709d09709c5889f9c7a42 into c2ca55c2c8dd40d4a037d1dd3a8c59e3(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 00:56:05,031 DEBUG [Listener at localhost/37869] regionserver.HRegion(2288): Compaction status journal for a55e824ba00709d09709c5889f9c7a42: 2023-02-13 00:56:05,034 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 00:56:05,036 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/1b93672aad624b66b5769229e62e5264, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/1c42c61791fe4961a182c5433f688472, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/e611d5031c1446a5acb151b004ef18a7, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/95798a2f2831432582bd36084d17c46b, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/e762d0689616462695e589a9fdd813f3] to archive 2023-02-13 00:56:05,036 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:05,037 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 00:56:05,038 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/1b93672aad624b66b5769229e62e5264 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/1b93672aad624b66b5769229e62e5264 2023-02-13 00:56:05,039 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/1c42c61791fe4961a182c5433f688472 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/1c42c61791fe4961a182c5433f688472 2023-02-13 00:56:05,040 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/e611d5031c1446a5acb151b004ef18a7 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/e611d5031c1446a5acb151b004ef18a7 2023-02-13 00:56:05,041 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 00:56:05,041 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/95798a2f2831432582bd36084d17c46b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/95798a2f2831432582bd36084d17c46b 2023-02-13 00:56:05,042 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 00:56:05,042 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/e762d0689616462695e589a9fdd813f3 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family2/e762d0689616462695e589a9fdd813f3 2023-02-13 00:56:05,046 INFO [Listener at localhost/37869] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-13 00:56:05,058 INFO [Listener at localhost/37869] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-02-13 00:56:05,061 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302132d85dc70c6a84b44b58796d8aaf700a9_del 2023-02-13 00:56:05,063 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230213f924c6c4cbfd42939034ef7cb43ead07_del 2023-02-13 00:56:05,065 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202302134f16ced26407438fbebbd2b9834ea796_del 2023-02-13 00:56:05,065 INFO [Listener at localhost/37869] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-02-13 00:56:05,066 DEBUG [Listener at localhost/37869] zookeeper.ReadOnlyZKClient(139): Connect 0x20d7c20f to 127.0.0.1:54478 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:56:05,071 DEBUG [Listener at localhost/37869] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7d841d14, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:56:05,072 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230213 for table='ns:testMinorCompaction' and column='family1' 2023-02-13 00:56:05,072 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230213 for table='ns:testMinorCompaction' and column='family1' 2023-02-13 00:56:05,072 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230213 for table='ns:testMinorCompaction' and column='family1' 2023-02-13 00:56:05,493 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213161336e84c20456cae2800dbc82b085e to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230213161336e84c20456cae2800dbc82b085e 2023-02-13 00:56:05,494 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230213 table='ns:testMinorCompaction' column='family1' 2023-02-13 00:56:05,495 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021356df1a96c60d42c7955031150295cab3 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023021356df1a96c60d42c7955031150295cab3 2023-02-13 00:56:05,496 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='ns:testMinorCompaction' column='family1' 2023-02-13 00:56:05,512 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:56:05,515 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59362, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:56:05,521 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 00:56:05,523 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40854, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 00:56:05,537 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230213/family1/440ea8f7ec9d4a399025d003771b6285 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-02-13 00:56:05,537 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230213/family1/440ea8f7ec9d4a399025d003771b6285}] 2023-02-13 00:56:05,540 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230213/family1/b3460abb6a1948f585055b104c0935d4 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-02-13 00:56:05,540 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230213/family1/b3460abb6a1948f585055b104c0935d4}] 2023-02-13 00:56:05,548 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230213/family1/b3460abb6a1948f585055b104c0935d4 for inclusion in a55e824ba00709d09709c5889f9c7a42/family1 2023-02-13 00:56:05,548 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230213/family1/440ea8f7ec9d4a399025d003771b6285 for inclusion in 914f10fbb08273a073f4cd678112ab73/family1 2023-02-13 00:56:05,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-02-13 00:56:05,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-13 00:56:05,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-02-13 00:56:05,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for a55e824ba00709d09709c5889f9c7a42: 2023-02-13 00:56:05,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-13 00:56:05,551 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for 914f10fbb08273a073f4cd678112ab73: 2023-02-13 00:56:05,551 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230213/family1/b3460abb6a1948f585055b104c0935d4 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__ns_testMinorCompaction__vuv8q22v9ib93806lrlok3fbp54fl967saiepb8n6phuv95a4535cavjovb8gve8/family1/b3460abb6a1948f585055b104c0935d4 2023-02-13 00:56:05,551 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230213/family1/440ea8f7ec9d4a399025d003771b6285 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__ns_testMinorCompaction__6erlj0757a8odqfaq7ij3tj4772u6shrvge56qelf6rv772lofl0ifkb2s2ve019/family1/440ea8f7ec9d4a399025d003771b6285 2023-02-13 00:56:05,553 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__ns_testMinorCompaction__vuv8q22v9ib93806lrlok3fbp54fl967saiepb8n6phuv95a4535cavjovb8gve8/family1/b3460abb6a1948f585055b104c0935d4 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/a2e1806c5bd6450495482cfec90bb4d2_SeqId_28_ 2023-02-13 00:56:05,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__ns_testMinorCompaction__6erlj0757a8odqfaq7ij3tj4772u6shrvge56qelf6rv772lofl0ifkb2s2ve019/family1/440ea8f7ec9d4a399025d003771b6285 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/73a4ee05e4074d8d8e85361ba79c8b76_SeqId_28_ 2023-02-13 00:56:05,554 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__ns_testMinorCompaction__vuv8q22v9ib93806lrlok3fbp54fl967saiepb8n6phuv95a4535cavjovb8gve8/family1/b3460abb6a1948f585055b104c0935d4 into a55e824ba00709d09709c5889f9c7a42/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/a2e1806c5bd6450495482cfec90bb4d2_SeqId_28_ - updating store file list. 2023-02-13 00:56:05,554 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__ns_testMinorCompaction__6erlj0757a8odqfaq7ij3tj4772u6shrvge56qelf6rv772lofl0ifkb2s2ve019/family1/440ea8f7ec9d4a399025d003771b6285 into 914f10fbb08273a073f4cd678112ab73/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/73a4ee05e4074d8d8e85361ba79c8b76_SeqId_28_ - updating store file list. 2023-02-13 00:56:05,571 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 00:56:05,571 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/a2e1806c5bd6450495482cfec90bb4d2_SeqId_28_ into a55e824ba00709d09709c5889f9c7a42/family1 2023-02-13 00:56:05,571 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6a875512 to 127.0.0.1:54478 2023-02-13 00:56:05,571 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__ns_testMinorCompaction__vuv8q22v9ib93806lrlok3fbp54fl967saiepb8n6phuv95a4535cavjovb8gve8/family1/b3460abb6a1948f585055b104c0935d4 into a55e824ba00709d09709c5889f9c7a42/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/family1/a2e1806c5bd6450495482cfec90bb4d2_SeqId_28_) 2023-02-13 00:56:05,572 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/73a4ee05e4074d8d8e85361ba79c8b76_SeqId_28_ into 914f10fbb08273a073f4cd678112ab73/family1 2023-02-13 00:56:05,571 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:56:05,572 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__ns_testMinorCompaction__6erlj0757a8odqfaq7ij3tj4772u6shrvge56qelf6rv772lofl0ifkb2s2ve019/family1/440ea8f7ec9d4a399025d003771b6285 into 914f10fbb08273a073f4cd678112ab73/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/family1/73a4ee05e4074d8d8e85361ba79c8b76_SeqId_28_) 2023-02-13 00:56:05,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__ns_testMinorCompaction__vuv8q22v9ib93806lrlok3fbp54fl967saiepb8n6phuv95a4535cavjovb8gve8/family1/b3460abb6a1948f585055b104c0935d4 2023-02-13 00:56:05,572 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 00:56:05,572 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x314f3d9c to 127.0.0.1:54478 2023-02-13 00:56:05,572 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:56:05,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__ns_testMinorCompaction__6erlj0757a8odqfaq7ij3tj4772u6shrvge56qelf6rv772lofl0ifkb2s2ve019/family1/440ea8f7ec9d4a399025d003771b6285 2023-02-13 00:56:05,574 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230213 table='ns:testMinorCompaction' column='family1' 2023-02-13 00:56:05,574 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='ns:testMinorCompaction' column='family1' 2023-02-13 00:56:05,575 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:05,575 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:05,576 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230213a9644f83cd8f49bda2d818643903d09b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230213a9644f83cd8f49bda2d818643903d09b 2023-02-13 00:56:05,576 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-13 00:56:05,577 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230213' 2023-02-13 00:56:05,577 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230213d64a499bf9124b2b87df7e9888c59188 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230213d64a499bf9124b2b87df7e9888c59188 2023-02-13 00:56:05,577 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-13 00:56:05,577 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230213' 2023-02-13 00:56:05,577 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230213 for table='ns:testMinorCompaction' and column='family1' 2023-02-13 00:56:05,577 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230213 for table='ns:testMinorCompaction' and column='family1' 2023-02-13 00:56:05,895 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213c506ed858a0b4a0f9027f4785494137f to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230213c506ed858a0b4a0f9027f4785494137f 2023-02-13 00:56:05,896 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='ns:testMinorCompaction' column='family1' 2023-02-13 00:56:05,918 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230213/family1/14addfda458541ffa8fc421e655a4e05 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-02-13 00:56:05,919 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230213/family1/14addfda458541ffa8fc421e655a4e05}] 2023-02-13 00:56:05,921 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230213/family1/14addfda458541ffa8fc421e655a4e05 for inclusion in 5bae73767db4da59da83135d4e2a56db/family1 2023-02-13 00:56:05,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-02-13 00:56:05,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(777): Region bounds: first= last=1 2023-02-13 00:56:05,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for 5bae73767db4da59da83135d4e2a56db: 2023-02-13 00:56:05,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230213/family1/14addfda458541ffa8fc421e655a4e05 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__ns_testMinorCompaction__et2m57ojvve7hhp1tv4d38mokcei6hs0na1a4fijq4jep9p5v0rmtkuo6emkt0mu/family1/14addfda458541ffa8fc421e655a4e05 2023-02-13 00:56:05,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__ns_testMinorCompaction__et2m57ojvve7hhp1tv4d38mokcei6hs0na1a4fijq4jep9p5v0rmtkuo6emkt0mu/family1/14addfda458541ffa8fc421e655a4e05 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/7fefa3e81cf54d07b021758715197a75_SeqId_28_ 2023-02-13 00:56:05,925 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__ns_testMinorCompaction__et2m57ojvve7hhp1tv4d38mokcei6hs0na1a4fijq4jep9p5v0rmtkuo6emkt0mu/family1/14addfda458541ffa8fc421e655a4e05 into 5bae73767db4da59da83135d4e2a56db/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/7fefa3e81cf54d07b021758715197a75_SeqId_28_ - updating store file list. 2023-02-13 00:56:05,927 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/7fefa3e81cf54d07b021758715197a75_SeqId_28_ into 5bae73767db4da59da83135d4e2a56db/family1 2023-02-13 00:56:05,927 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__ns_testMinorCompaction__et2m57ojvve7hhp1tv4d38mokcei6hs0na1a4fijq4jep9p5v0rmtkuo6emkt0mu/family1/14addfda458541ffa8fc421e655a4e05 into 5bae73767db4da59da83135d4e2a56db/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/family1/7fefa3e81cf54d07b021758715197a75_SeqId_28_) 2023-02-13 00:56:05,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__ns_testMinorCompaction__et2m57ojvve7hhp1tv4d38mokcei6hs0na1a4fijq4jep9p5v0rmtkuo6emkt0mu/family1/14addfda458541ffa8fc421e655a4e05 2023-02-13 00:56:05,929 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='ns:testMinorCompaction' column='family1' 2023-02-13 00:56:05,930 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:05,931 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230213c7b55ce04a844ebeae367a506c12bdc4 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230213c7b55ce04a844ebeae367a506c12bdc4 2023-02-13 00:56:05,931 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-13 00:56:05,931 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230213' 2023-02-13 00:56:05,931 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230213 for table='ns:testMinorCompaction' and column='family1' 2023-02-13 00:56:05,931 INFO [Listener at localhost/37869] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-02-13 00:56:05,933 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 00:56:05,935 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 00:56:05,937 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 00:56:05,938 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 00:56:05,953 INFO [Listener at localhost/37869] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=248 (was 284), OpenFileDescriptor=624 (was 752), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=122 (was 142), ProcessCount=167 (was 167), AvailableMemoryMB=3304 (was 3319) 2023-02-13 00:56:05,963 INFO [Listener at localhost/37869] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=248, OpenFileDescriptor=624, MaxFileDescriptor=60000, SystemLoadAverage=122, ProcessCount=167, AvailableMemoryMB=3303 2023-02-13 00:56:05,964 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-13 00:56:05,965 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:05,966 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 00:56:05,966 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-02-13 00:56:05,967 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 00:56:05,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-13 00:56:05,968 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96 2023-02-13 00:56:05,968 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96 empty. 2023-02-13 00:56:05,968 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96 2023-02-13 00:56:05,968 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-02-13 00:56:06,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-13 00:56:06,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-13 00:56:06,374 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-13 00:56:06,375 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 742bdf451b644123e692d39c2d4cda96, NAME => 'testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp 2023-02-13 00:56:06,569 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-13 00:56:06,780 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:56:06,781 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1603): Closing 742bdf451b644123e692d39c2d4cda96, disabling compactions & flushes 2023-02-13 00:56:06,781 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:06,781 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:06,781 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. after waiting 0 ms 2023-02-13 00:56:06,781 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:06,781 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1837): Closed testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:06,781 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 742bdf451b644123e692d39c2d4cda96: Waiting for close lock at 1676249766781Disabling compacts and flushes for region at 1676249766781Disabling writes for close at 1676249766781Writing region close event to WAL at 1676249766781Closed at 1676249766781 2023-02-13 00:56:06,783 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 00:56:06,784 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676249766783"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249766783"}]},"ts":"1676249766783"} 2023-02-13 00:56:06,785 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 00:56:06,785 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 00:56:06,785 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249766785"}]},"ts":"1676249766785"} 2023-02-13 00:56:06,786 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-02-13 00:56:06,791 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=742bdf451b644123e692d39c2d4cda96, ASSIGN}] 2023-02-13 00:56:06,792 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=742bdf451b644123e692d39c2d4cda96, ASSIGN 2023-02-13 00:56:06,793 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=742bdf451b644123e692d39c2d4cda96, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=false 2023-02-13 00:56:06,944 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=742bdf451b644123e692d39c2d4cda96, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:06,944 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676249766944"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249766944"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249766944"}]},"ts":"1676249766944"} 2023-02-13 00:56:06,946 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure 742bdf451b644123e692d39c2d4cda96, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:56:07,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-13 00:56:07,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:07,099 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 742bdf451b644123e692d39c2d4cda96, NAME => 'testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:56:07,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy 742bdf451b644123e692d39c2d4cda96 2023-02-13 00:56:07,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:56:07,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 742bdf451b644123e692d39c2d4cda96 2023-02-13 00:56:07,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 742bdf451b644123e692d39c2d4cda96 2023-02-13 00:56:07,101 INFO [StoreOpener-742bdf451b644123e692d39c2d4cda96-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 742bdf451b644123e692d39c2d4cda96 2023-02-13 00:56:07,102 DEBUG [StoreOpener-742bdf451b644123e692d39c2d4cda96-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1 2023-02-13 00:56:07,102 DEBUG [StoreOpener-742bdf451b644123e692d39c2d4cda96-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1 2023-02-13 00:56:07,102 INFO [StoreOpener-742bdf451b644123e692d39c2d4cda96-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 742bdf451b644123e692d39c2d4cda96 columnFamilyName family1 2023-02-13 00:56:07,103 INFO [StoreOpener-742bdf451b644123e692d39c2d4cda96-1] regionserver.HStore(310): Store=742bdf451b644123e692d39c2d4cda96/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:07,104 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96 2023-02-13 00:56:07,104 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96 2023-02-13 00:56:07,106 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 742bdf451b644123e692d39c2d4cda96 2023-02-13 00:56:07,108 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:56:07,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 742bdf451b644123e692d39c2d4cda96; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69652621, jitterRate=0.03790493309497833}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 00:56:07,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 742bdf451b644123e692d39c2d4cda96: Running coprocessor pre-open hook at 1676249767100Writing region info on filesystem at 1676249767100Initializing all the Stores at 1676249767100Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1676249767100Cleaning up temporary data from old regions at 1676249767105 (+5 ms)Cleaning up detritus from prior splits at 1676249767105Running coprocessor post-open hooks at 1676249767109 (+4 ms)Region opened successfully at 1676249767109 2023-02-13 00:56:07,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96., pid=114, masterSystemTime=1676249767097 2023-02-13 00:56:07,111 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:07,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:07,111 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=742bdf451b644123e692d39c2d4cda96, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:07,111 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676249767111"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249767111"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249767111"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249767111"}]},"ts":"1676249767111"} 2023-02-13 00:56:07,113 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-13 00:56:07,113 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure 742bdf451b644123e692d39c2d4cda96, server=jenkins-hbase4.apache.org,36605,1676249622318 in 166 msec 2023-02-13 00:56:07,115 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-13 00:56:07,115 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=742bdf451b644123e692d39c2d4cda96, ASSIGN in 322 msec 2023-02-13 00:56:07,115 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 00:56:07,115 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249767115"}]},"ts":"1676249767115"} 2023-02-13 00:56:07,116 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-02-13 00:56:07,118 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 00:56:07,119 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 1.1540 sec 2023-02-13 00:56:07,614 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 00:56:08,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-13 00:56:08,071 INFO [Listener at localhost/37869] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-02-13 00:56:08,091 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:08,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,092 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 00:56:08,092 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:08,092 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 00:56:08,092 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 00:56:08,093 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,093 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,094 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:08,094 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:08,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:08,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:08,094 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:08,095 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:08,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,095 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:08,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,096 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,096 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,096 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 00:56:08,096 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:08,096 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 00:56:08,099 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:08,099 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:08,100 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:08,100 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. started... 2023-02-13 00:56:08,100 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 742bdf451b644123e692d39c2d4cda96 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-13 00:56:08,523 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113049321e6ad5724669a31a7a20ac4f57ed to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113049321e6ad5724669a31a7a20ac4f57ed 2023-02-13 00:56:08,926 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/.tmp/family1/794c0a2a6d4143888ad413bbbb7f9d55 2023-02-13 00:56:08,929 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/.tmp/family1/794c0a2a6d4143888ad413bbbb7f9d55 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/794c0a2a6d4143888ad413bbbb7f9d55 2023-02-13 00:56:08,932 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/794c0a2a6d4143888ad413bbbb7f9d55, entries=1000, sequenceid=5, filesize=166.0 K 2023-02-13 00:56:08,933 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 742bdf451b644123e692d39c2d4cda96 in 833ms, sequenceid=5, compaction requested=false 2023-02-13 00:56:08,934 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 742bdf451b644123e692d39c2d4cda96: 2023-02-13 00:56:08,934 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:08,934 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:08,934 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:08,934 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:08,934 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 00:56:08,934 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 00:56:08,935 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,935 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:08,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:08,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:08,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:08,936 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,936 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:08,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:08,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:08,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:08,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:08,937 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 00:56:08,937 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6b640e5a[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:08,937 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 00:56:08,937 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,939 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,939 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 00:56:08,939 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 00:56:08,939 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:56:08,939 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:08,939 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:08,940 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 00:56:08,940 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:08,940 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:08,940 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 00:56:08,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:08,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:08,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:08,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:08,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:08,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:08,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:08,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:08,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:56:08,942 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:56:08,942 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 00:56:08,942 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:08,943 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:08,943 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,943 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:08,944 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,944 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:08,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:08,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,944 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:08,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:08,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:08,944 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:08,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:08,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:08,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,946 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:08,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:08,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:08,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:08,948 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:08,948 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:08,948 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:08,948 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:08,948 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 00:56:08,948 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:08,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:08,948 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,948 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:08,948 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:08,948 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,948 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,948 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:56:08,948 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:08,948 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:08,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:08,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:09,048 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:09,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:09,052 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:09,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:09,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,054 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 00:56:09,054 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:09,054 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 00:56:09,054 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 00:56:09,054 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,054 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,056 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:09,056 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:09,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:09,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:09,056 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:09,056 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:09,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,057 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:09,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,057 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,057 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 00:56:09,057 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,057 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 00:56:09,057 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:09,058 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 00:56:09,058 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:09,058 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:09,058 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:09,058 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. started... 2023-02-13 00:56:09,058 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 742bdf451b644123e692d39c2d4cda96 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-13 00:56:09,469 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112821995ba567f7497fb6cc7e88dec5517d to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112821995ba567f7497fb6cc7e88dec5517d 2023-02-13 00:56:09,873 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/.tmp/family1/11c5a90132c14a69a638e9bfb41ce886 2023-02-13 00:56:09,877 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/.tmp/family1/11c5a90132c14a69a638e9bfb41ce886 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/11c5a90132c14a69a638e9bfb41ce886 2023-02-13 00:56:09,880 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/11c5a90132c14a69a638e9bfb41ce886, entries=1, sequenceid=9, filesize=5.0 K 2023-02-13 00:56:09,881 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 742bdf451b644123e692d39c2d4cda96 in 823ms, sequenceid=9, compaction requested=false 2023-02-13 00:56:09,881 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 742bdf451b644123e692d39c2d4cda96: 2023-02-13 00:56:09,881 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:09,881 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:09,881 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:09,881 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:09,881 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 00:56:09,881 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 00:56:09,882 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:09,882 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:09,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:09,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:09,883 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:09,883 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:09,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:09,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:09,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:09,884 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 00:56:09,884 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7848ae23[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:09,884 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 00:56:09,884 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,885 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,885 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 00:56:09,885 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 00:56:09,885 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:09,885 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:09,885 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:56:09,886 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:09,886 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 00:56:09,886 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:09,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:09,886 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 00:56:09,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:09,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:09,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:09,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:09,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:09,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:09,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:09,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:56:09,888 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:56:09,888 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 00:56:09,888 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:09,888 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:09,888 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,888 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:09,891 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,891 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:09,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:09,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,891 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:09,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:09,891 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:09,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:09,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:09,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:09,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,892 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:09,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:09,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:09,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:09,895 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:09,895 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,895 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,895 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:09,895 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,895 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:09,895 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 00:56:09,895 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:09,895 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:09,895 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 00:56:09,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:09,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:09,895 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,895 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:56:09,895 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:09,895 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:09,895 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:09,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:09,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:09,996 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:09,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:09,999 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:10,001 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:10,001 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,001 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 00:56:10,001 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:10,001 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 00:56:10,001 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 00:56:10,001 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,001 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,005 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:10,005 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:10,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:10,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:10,005 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:10,005 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:10,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,006 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:10,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,006 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,006 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 00:56:10,006 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,006 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 00:56:10,006 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:10,006 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 00:56:10,006 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:10,006 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:10,006 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:10,006 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. started... 2023-02-13 00:56:10,007 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 742bdf451b644123e692d39c2d4cda96 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:56:10,416 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512016d3fd612122f4de1846982a9b0572759 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512016d3fd612122f4de1846982a9b0572759 2023-02-13 00:56:10,821 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/.tmp/family1/584fad3180d34cee8d280a06d14d0ff4 2023-02-13 00:56:10,826 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/.tmp/family1/584fad3180d34cee8d280a06d14d0ff4 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/584fad3180d34cee8d280a06d14d0ff4 2023-02-13 00:56:10,829 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/584fad3180d34cee8d280a06d14d0ff4, entries=1, sequenceid=13, filesize=4.9 K 2023-02-13 00:56:10,829 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 742bdf451b644123e692d39c2d4cda96 in 822ms, sequenceid=13, compaction requested=true 2023-02-13 00:56:10,830 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 742bdf451b644123e692d39c2d4cda96: 2023-02-13 00:56:10,830 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:10,830 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:10,830 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:10,830 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:10,830 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 00:56:10,830 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 00:56:10,832 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,832 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:10,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:10,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:10,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:10,832 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,832 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:10,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:10,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:10,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:10,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:10,834 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 00:56:10,834 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4bf292ab[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:10,834 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 00:56:10,834 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,835 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,835 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 00:56:10,835 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 00:56:10,835 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:56:10,835 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:10,835 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:10,836 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:10,836 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 00:56:10,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:10,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:10,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:10,836 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:10,837 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 00:56:10,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:10,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:10,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:10,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:10,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:10,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:56:10,838 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:56:10,838 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 00:56:10,838 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:10,838 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:10,838 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,838 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:10,840 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,840 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:10,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:10,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,840 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:10,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:10,840 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:10,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:10,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:10,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:10,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:10,841 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:10,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:10,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:10,844 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:10,844 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:10,844 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:10,844 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 00:56:10,844 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:10,844 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:10,844 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,844 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:10,844 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:10,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:10,844 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,844 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:56:10,844 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:10,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:10,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:10,844 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,845 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,945 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:10,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:10,948 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:10,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:10,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,950 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 00:56:10,950 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:10,950 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 00:56:10,950 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 00:56:10,950 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,950 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,952 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:10,952 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:10,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:10,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:10,952 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:10,952 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:10,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,952 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:10,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,953 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,953 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 00:56:10,953 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:10,953 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 00:56:10,953 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:10,953 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 00:56:10,953 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:10,953 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:10,953 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:10,953 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. started... 2023-02-13 00:56:10,953 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 742bdf451b644123e692d39c2d4cda96 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:56:11,363 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120509a4853cad694b519af9a21802c54237 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120509a4853cad694b519af9a21802c54237 2023-02-13 00:56:11,768 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/.tmp/family1/54af0e01ac494892ae515295e2f7b654 2023-02-13 00:56:11,772 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/.tmp/family1/54af0e01ac494892ae515295e2f7b654 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/54af0e01ac494892ae515295e2f7b654 2023-02-13 00:56:11,775 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/54af0e01ac494892ae515295e2f7b654, entries=1, sequenceid=17, filesize=4.9 K 2023-02-13 00:56:11,776 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 742bdf451b644123e692d39c2d4cda96 in 823ms, sequenceid=17, compaction requested=true 2023-02-13 00:56:11,776 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 742bdf451b644123e692d39c2d4cda96: 2023-02-13 00:56:11,776 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:11,776 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:11,776 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:11,776 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:11,776 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 00:56:11,776 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 00:56:11,778 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:11,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:11,778 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:11,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:11,778 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:11,779 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:11,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:11,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:11,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:11,780 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 00:56:11,780 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5662de85[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:11,780 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 00:56:11,780 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,781 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,781 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 00:56:11,781 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 00:56:11,781 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:11,781 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:11,781 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:56:11,784 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 00:56:11,784 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:11,784 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:11,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:11,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:11,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:11,784 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 00:56:11,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:11,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:11,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:11,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:11,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:11,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:56:11,786 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:56:11,786 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 00:56:11,786 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:11,786 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:11,786 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,786 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:11,787 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,787 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:11,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:11,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,787 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:11,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:11,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:11,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:11,787 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:11,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:11,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,788 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:11,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:11,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:11,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:11,791 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:11,791 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,791 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:11,791 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:11,791 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 00:56:11,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:11,791 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 00:56:11,791 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:11,791 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:11,791 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:11,791 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:11,791 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:11,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:11,791 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,791 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:56:11,791 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,791 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:11,891 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:11,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:11,896 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:11,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:11,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,897 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 00:56:11,897 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:11,897 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 00:56:11,897 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 00:56:11,898 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,898 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,899 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:11,899 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:11,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:11,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:11,899 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:11,899 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:11,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,900 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:11,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,900 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,900 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 00:56:11,900 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:11,900 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 00:56:11,900 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:11,900 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 00:56:11,901 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:11,901 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:11,901 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:11,901 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. started... 2023-02-13 00:56:11,901 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 742bdf451b644123e692d39c2d4cda96 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:56:12,310 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122808bfddc357ae4c3391ac8d288b955539 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015122808bfddc357ae4c3391ac8d288b955539 2023-02-13 00:56:12,635 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 00:56:12,715 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/.tmp/family1/fecc47d07c7245b495da0de74b854803 2023-02-13 00:56:12,719 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/.tmp/family1/fecc47d07c7245b495da0de74b854803 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/fecc47d07c7245b495da0de74b854803 2023-02-13 00:56:12,723 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/fecc47d07c7245b495da0de74b854803, entries=1, sequenceid=21, filesize=4.9 K 2023-02-13 00:56:12,724 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 742bdf451b644123e692d39c2d4cda96 in 823ms, sequenceid=21, compaction requested=true 2023-02-13 00:56:12,724 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 742bdf451b644123e692d39c2d4cda96: 2023-02-13 00:56:12,724 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:12,724 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:12,724 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:12,724 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:12,724 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 00:56:12,724 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 00:56:12,727 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:12,727 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:12,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:12,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:12,728 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:12,728 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:12,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:12,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:12,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:12,740 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 00:56:12,740 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1d3f0ccd[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:12,740 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 00:56:12,741 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,742 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,742 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 00:56:12,742 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 00:56:12,742 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:12,742 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:12,742 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:56:12,743 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 00:56:12,743 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:12,744 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:12,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:12,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:12,744 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 00:56:12,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:12,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:12,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:12,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:12,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:12,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:12,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:56:12,747 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:56:12,747 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 00:56:12,747 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:12,747 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:12,747 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,747 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:12,748 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,749 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:12,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:12,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,749 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:12,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,749 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:12,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:12,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:12,751 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:12,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:12,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:12,758 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,758 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:12,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,759 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:12,758 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,759 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 00:56:12,759 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,759 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:12,759 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,759 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,759 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,758 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,759 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:12,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,759 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:56:12,759 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:12,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,860 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:12,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:12,864 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:12,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:12,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,865 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 00:56:12,865 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:12,865 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 00:56:12,865 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 00:56:12,866 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,866 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,867 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:12,867 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,867 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:12,867 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:12,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,868 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:12,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,869 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,869 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 00:56:12,870 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:12,870 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 00:56:12,870 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:12,870 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 00:56:12,870 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:12,870 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:12,870 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:12,870 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. started... 2023-02-13 00:56:12,870 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 742bdf451b644123e692d39c2d4cda96 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:56:12,881 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512318bf236815d9e4a4cae7588e57287c8b8 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512318bf236815d9e4a4cae7588e57287c8b8 2023-02-13 00:56:13,287 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/.tmp/family1/09d5e29db4a64113970c7a8435681cbe 2023-02-13 00:56:13,291 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/.tmp/family1/09d5e29db4a64113970c7a8435681cbe as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/09d5e29db4a64113970c7a8435681cbe 2023-02-13 00:56:13,293 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/09d5e29db4a64113970c7a8435681cbe, entries=1, sequenceid=25, filesize=4.9 K 2023-02-13 00:56:13,294 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 742bdf451b644123e692d39c2d4cda96 in 424ms, sequenceid=25, compaction requested=true 2023-02-13 00:56:13,294 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 742bdf451b644123e692d39c2d4cda96: 2023-02-13 00:56:13,294 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:13,294 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:13,294 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:13,294 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,294 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 00:56:13,294 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 00:56:13,296 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,296 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:13,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:13,296 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,296 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:13,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:13,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:13,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:13,297 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 00:56:13,297 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@203b90ff[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:13,297 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 00:56:13,297 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,298 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,298 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 00:56:13,298 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 00:56:13,298 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:56:13,298 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,298 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:13,299 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 00:56:13,299 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,299 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:13,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:13,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:13,300 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 00:56:13,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:13,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:13,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:13,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:56:13,301 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:56:13,301 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 00:56:13,301 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:13,301 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:13,302 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,302 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:13,304 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,304 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:13,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:13,304 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:13,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:13,305 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:13,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:13,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:13,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,306 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:13,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,308 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,308 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:13,308 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:13,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:13,308 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:13,308 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 00:56:13,308 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:13,308 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:13,308 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,308 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:13,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:13,308 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,309 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:56:13,309 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:13,309 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,309 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,409 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:13,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:13,412 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:13,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:13,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,414 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 00:56:13,414 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:13,414 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 00:56:13,414 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 00:56:13,414 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,414 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,416 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:13,416 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:13,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:13,416 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,416 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:13,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,416 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:13,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,417 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,417 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 00:56:13,417 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,417 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 00:56:13,417 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:13,417 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 00:56:13,417 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:13,418 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:13,418 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:13,418 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. started... 2023-02-13 00:56:13,418 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 742bdf451b644123e692d39c2d4cda96 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:56:13,829 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101b5994f388754439bb9bfecea66af977b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101b5994f388754439bb9bfecea66af977b 2023-02-13 00:56:13,833 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/.tmp/family1/ad3ff60da34c4c63b9e04e637cb98b12 2023-02-13 00:56:13,837 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/.tmp/family1/ad3ff60da34c4c63b9e04e637cb98b12 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/ad3ff60da34c4c63b9e04e637cb98b12 2023-02-13 00:56:13,839 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/ad3ff60da34c4c63b9e04e637cb98b12, entries=1, sequenceid=29, filesize=4.9 K 2023-02-13 00:56:13,840 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 742bdf451b644123e692d39c2d4cda96 in 422ms, sequenceid=29, compaction requested=true 2023-02-13 00:56:13,840 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 742bdf451b644123e692d39c2d4cda96: 2023-02-13 00:56:13,840 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:13,840 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:13,840 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:13,840 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,840 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 00:56:13,841 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 00:56:13,842 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,842 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:13,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:13,843 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,843 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:13,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:13,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:13,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:13,844 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 00:56:13,844 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7f832999[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:13,844 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 00:56:13,844 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,845 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,845 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 00:56:13,845 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 00:56:13,845 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:56:13,845 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,845 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:13,846 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 00:56:13,846 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:13,846 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,846 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 00:56:13,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:13,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:13,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:13,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:13,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:13,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:56:13,848 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:56:13,848 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 00:56:13,848 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:13,848 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:13,848 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,848 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:13,849 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,849 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:13,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:13,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,849 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:13,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:13,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:13,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:13,850 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:13,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,851 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:13,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,854 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,854 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:13,854 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:13,854 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:13,854 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 00:56:13,854 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 00:56:13,854 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:13,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:13,854 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,854 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:13,854 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,854 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,854 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,854 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,854 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:56:13,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:13,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:13,854 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:13,954 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:13,955 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:13,960 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:13,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:13,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,961 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 00:56:13,961 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:13,961 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 00:56:13,961 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 00:56:13,962 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,962 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,963 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,963 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:13,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:13,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:13,963 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:13,963 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:13,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,964 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:13,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,964 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,964 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 00:56:13,964 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:13,964 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 00:56:13,964 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:13,964 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 00:56:13,964 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:13,964 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:13,964 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:13,965 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. started... 2023-02-13 00:56:13,965 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 742bdf451b644123e692d39c2d4cda96 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:56:14,375 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601030c5b9b2c71064528913c6f31821e5f9a to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601030c5b9b2c71064528913c6f31821e5f9a 2023-02-13 00:56:14,380 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/.tmp/family1/7e3064fa3d30457ba27924d79a79cfd3 2023-02-13 00:56:14,384 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/.tmp/family1/7e3064fa3d30457ba27924d79a79cfd3 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/7e3064fa3d30457ba27924d79a79cfd3 2023-02-13 00:56:14,386 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/7e3064fa3d30457ba27924d79a79cfd3, entries=1, sequenceid=33, filesize=4.9 K 2023-02-13 00:56:14,387 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 742bdf451b644123e692d39c2d4cda96 in 422ms, sequenceid=33, compaction requested=true 2023-02-13 00:56:14,387 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 742bdf451b644123e692d39c2d4cda96: 2023-02-13 00:56:14,387 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:14,387 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:14,387 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:14,387 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:14,387 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 00:56:14,387 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 00:56:14,389 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,389 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:14,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:14,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:14,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:14,389 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,389 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:14,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:14,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:14,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:14,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:14,390 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 00:56:14,390 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3fd5461a[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:14,390 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 00:56:14,390 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,392 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,392 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 00:56:14,392 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 00:56:14,392 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:56:14,392 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:14,392 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:14,393 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:14,393 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 00:56:14,393 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:14,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:14,393 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 00:56:14,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:14,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:14,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:14,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:14,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:14,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:14,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:14,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:56:14,396 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:56:14,396 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 00:56:14,396 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:14,396 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:14,396 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,396 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:14,397 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,397 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:14,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:14,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,397 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:14,398 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:14,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:14,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,399 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:14,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:14,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:14,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:14,412 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:14,412 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,412 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,412 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:14,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,412 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,412 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,412 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:14,412 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,412 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 00:56:14,412 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:14,412 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,412 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,412 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 00:56:14,412 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:56:14,413 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:14,513 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:14,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:14,517 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:14,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:14,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,518 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 00:56:14,518 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:14,518 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 00:56:14,518 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 00:56:14,519 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,519 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,520 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:14,520 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,520 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:14,520 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:14,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,520 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:14,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,521 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,521 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 00:56:14,521 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:14,521 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 00:56:14,521 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:14,521 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 00:56:14,521 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:14,521 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:14,521 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:14,521 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. started... 2023-02-13 00:56:14,521 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 742bdf451b644123e692d39c2d4cda96 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:56:14,932 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509079ff08573393b42b8b203734621e4e034 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201509079ff08573393b42b8b203734621e4e034 2023-02-13 00:56:15,337 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/.tmp/family1/0e4bcbbf129c4364858a4c3e60f8017a 2023-02-13 00:56:15,340 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/.tmp/family1/0e4bcbbf129c4364858a4c3e60f8017a as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/0e4bcbbf129c4364858a4c3e60f8017a 2023-02-13 00:56:15,343 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/0e4bcbbf129c4364858a4c3e60f8017a, entries=1, sequenceid=37, filesize=4.9 K 2023-02-13 00:56:15,344 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 742bdf451b644123e692d39c2d4cda96 in 823ms, sequenceid=37, compaction requested=true 2023-02-13 00:56:15,344 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 742bdf451b644123e692d39c2d4cda96: 2023-02-13 00:56:15,344 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:15,344 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:15,344 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:15,344 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:15,344 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 00:56:15,344 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 00:56:15,346 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,346 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:15,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:15,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:15,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:15,346 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,346 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:15,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:15,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:15,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:15,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:15,347 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 00:56:15,347 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5ae96c49[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:15,347 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 00:56:15,347 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,349 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,349 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 00:56:15,349 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 00:56:15,349 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:56:15,349 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:15,350 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:15,351 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 00:56:15,351 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:15,351 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:15,351 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 00:56:15,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:15,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:15,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:15,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:15,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:15,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:15,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:15,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:15,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:56:15,353 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:56:15,353 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 00:56:15,353 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:15,353 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:15,353 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,353 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:15,354 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,354 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:15,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:15,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,354 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:15,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:15,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:15,354 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:15,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:15,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:15,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:15,355 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:15,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:15,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:15,357 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:15,357 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,357 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:15,357 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:15,357 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:15,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:15,357 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 00:56:15,357 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:15,357 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 00:56:15,357 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:15,357 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:15,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:15,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:15,358 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,358 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,358 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,358 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:56:15,358 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:15,458 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:15,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:15,462 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:15,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:15,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,463 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 00:56:15,463 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:15,463 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 00:56:15,463 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 00:56:15,464 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,464 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,465 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:15,465 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:15,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:15,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:15,466 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:15,466 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:15,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,466 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:15,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,466 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,467 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 00:56:15,467 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:15,467 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 00:56:15,467 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:15,467 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 00:56:15,467 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:15,467 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:15,467 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:15,467 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. started... 2023-02-13 00:56:15,467 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 742bdf451b644123e692d39c2d4cda96 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 00:56:15,876 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511202eb114f8812445f396a1907ed8fc0e2d to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511202eb114f8812445f396a1907ed8fc0e2d 2023-02-13 00:56:16,281 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/.tmp/family1/50d2eb3851ef4620805580d8544d1942 2023-02-13 00:56:16,285 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/.tmp/family1/50d2eb3851ef4620805580d8544d1942 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/50d2eb3851ef4620805580d8544d1942 2023-02-13 00:56:16,288 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/50d2eb3851ef4620805580d8544d1942, entries=1, sequenceid=41, filesize=4.9 K 2023-02-13 00:56:16,288 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 742bdf451b644123e692d39c2d4cda96 in 821ms, sequenceid=41, compaction requested=true 2023-02-13 00:56:16,288 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 742bdf451b644123e692d39c2d4cda96: 2023-02-13 00:56:16,288 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:16,288 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:16,289 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:16,289 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:16,289 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 00:56:16,289 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 00:56:16,290 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:16,290 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:16,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:16,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:16,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:16,290 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:16,290 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:16,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:16,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:16,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:16,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:16,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:16,291 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 00:56:16,292 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2629a383[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:16,292 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 00:56:16,292 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:16,293 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:16,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:16,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:16,293 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 00:56:16,293 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 00:56:16,293 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:16,293 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:16,293 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:56:16,294 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 00:56:16,294 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:16,294 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:16,294 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 00:56:16,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:16,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:16,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:16,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:16,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:16,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:16,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:16,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:16,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:16,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:16,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:56:16,296 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 00:56:16,296 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 00:56:16,296 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:16,296 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:16,296 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:16,296 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:16,298 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:16,298 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:16,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:16,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:16,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:16,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:16,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:16,298 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:16,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:16,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:16,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:16,299 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:16,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:16,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:16,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:16,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:16,300 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:16,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:16,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:16,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:16,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:16,301 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:16,301 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:16,302 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:16,302 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:16,302 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 00:56:16,302 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:16,302 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 00:56:16,302 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:16,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:16,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:16,302 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:16,302 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:16,302 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:16,302 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:16,302 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:16,302 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 00:56:16,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:16,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:16,302 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:16,402 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:16,403 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:16,460 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43563] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-02-13 00:56:16,462 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] procedure2.ProcedureExecutor(1029): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 00:56:16,462 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 00:56:16,463 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:56:16,463 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676250376463 2023-02-13 00:56:16,464 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-13 00:56:16,465 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-02-13 00:56:16,465 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 00:56:16,465 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] zookeeper.ReadOnlyZKClient(139): Connect 0x079506e4 to 127.0.0.1:54478 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:56:16,475 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2fb6ff3a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:56:16,475 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 00:56:17,303 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601030395d3e7e103477fbc48a119b11ed690 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601030395d3e7e103477fbc48a119b11ed690 2023-02-13 00:56:17,304 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 00:56:17,316 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:56:17,318 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49838, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:56:17,323 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 00:56:17,324 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45006, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 00:56:17,331 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/c7148fd8007b42368fdb444214d4fdd2 first=Optional[r5] last=Optional[r6] 2023-02-13 00:56:17,332 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/c7148fd8007b42368fdb444214d4fdd2}] 2023-02-13 00:56:17,335 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/c7148fd8007b42368fdb444214d4fdd2 for inclusion in 742bdf451b644123e692d39c2d4cda96/family1 2023-02-13 00:56:17,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-02-13 00:56:17,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 00:56:17,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for 742bdf451b644123e692d39c2d4cda96: 2023-02-13 00:56:17,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/c7148fd8007b42368fdb444214d4fdd2 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__nc5rifc2uljqd7mhpqt1ed4ns56lpaqiej9gvm53r1tbkidgno68q5g7cv574rt5/family1/c7148fd8007b42368fdb444214d4fdd2 2023-02-13 00:56:17,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__nc5rifc2uljqd7mhpqt1ed4ns56lpaqiej9gvm53r1tbkidgno68q5g7cv574rt5/family1/c7148fd8007b42368fdb444214d4fdd2 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/ee7f2534644b4569ba42766862386f47_SeqId_44_ 2023-02-13 00:56:17,339 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__nc5rifc2uljqd7mhpqt1ed4ns56lpaqiej9gvm53r1tbkidgno68q5g7cv574rt5/family1/c7148fd8007b42368fdb444214d4fdd2 into 742bdf451b644123e692d39c2d4cda96/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/ee7f2534644b4569ba42766862386f47_SeqId_44_ - updating store file list. 2023-02-13 00:56:17,342 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/ee7f2534644b4569ba42766862386f47_SeqId_44_ into 742bdf451b644123e692d39c2d4cda96/family1 2023-02-13 00:56:17,342 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__nc5rifc2uljqd7mhpqt1ed4ns56lpaqiej9gvm53r1tbkidgno68q5g7cv574rt5/family1/c7148fd8007b42368fdb444214d4fdd2 into 742bdf451b644123e692d39c2d4cda96/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/ee7f2534644b4569ba42766862386f47_SeqId_44_) 2023-02-13 00:56:17,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__nc5rifc2uljqd7mhpqt1ed4ns56lpaqiej9gvm53r1tbkidgno68q5g7cv574rt5/family1/c7148fd8007b42368fdb444214d4fdd2 2023-02-13 00:56:17,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-13 00:56:17,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-13 00:56:17,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(1912): 742bdf451b644123e692d39c2d4cda96/family1 is initiating minor compaction 2023-02-13 00:56:17,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@364c37d9; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:56:17,344 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 742bdf451b644123e692d39c2d4cda96/family1 in testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:17,344 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/11c5a90132c14a69a638e9bfb41ce886, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/584fad3180d34cee8d280a06d14d0ff4, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/54af0e01ac494892ae515295e2f7b654, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/fecc47d07c7245b495da0de74b854803, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/09d5e29db4a64113970c7a8435681cbe, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/ad3ff60da34c4c63b9e04e637cb98b12, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/7e3064fa3d30457ba27924d79a79cfd3, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/0e4bcbbf129c4364858a4c3e60f8017a, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/50d2eb3851ef4620805580d8544d1942, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/ee7f2534644b4569ba42766862386f47_SeqId_44_] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/.tmp, totalSize=49.7 K 2023-02-13 00:56:17,344 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 11c5a90132c14a69a638e9bfb41ce886, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-13 00:56:17,344 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 584fad3180d34cee8d280a06d14d0ff4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-13 00:56:17,344 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 54af0e01ac494892ae515295e2f7b654, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-13 00:56:17,344 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting fecc47d07c7245b495da0de74b854803, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-13 00:56:17,345 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 09d5e29db4a64113970c7a8435681cbe, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-13 00:56:17,345 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting ad3ff60da34c4c63b9e04e637cb98b12, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-13 00:56:17,345 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 7e3064fa3d30457ba27924d79a79cfd3, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-13 00:56:17,345 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 0e4bcbbf129c4364858a4c3e60f8017a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-13 00:56:17,345 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 50d2eb3851ef4620805580d8544d1942, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-13 00:56:17,345 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting ee7f2534644b4569ba42766862386f47_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-13 00:56:17,346 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 00:56:17,346 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:17,347 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101b5994f388754439bb9bfecea66af977b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101b5994f388754439bb9bfecea66af977b 2023-02-13 00:56:17,348 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601030c5b9b2c71064528913c6f31821e5f9a to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601030c5b9b2c71064528913c6f31821e5f9a 2023-02-13 00:56:17,348 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-02-13 00:56:17,348 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-02-13 00:56:17,349 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 00:56:17,349 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 00:56:17,373 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] throttle.PressureAwareThroughputController(145): 742bdf451b644123e692d39c2d4cda96#family1#compaction#165 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:56:17,390 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/.tmp/family1/de3f2775949b497fb0198ea52c95a901 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/de3f2775949b497fb0198ea52c95a901 2023-02-13 00:56:17,394 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 742bdf451b644123e692d39c2d4cda96/family1 of 742bdf451b644123e692d39c2d4cda96 into de3f2775949b497fb0198ea52c95a901(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:56:17,394 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 742bdf451b644123e692d39c2d4cda96: 2023-02-13 00:56:17,394 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1676249777344; duration=0sec 2023-02-13 00:56:17,394 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:56:17,436 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/11c5a90132c14a69a638e9bfb41ce886, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/584fad3180d34cee8d280a06d14d0ff4, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/54af0e01ac494892ae515295e2f7b654, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/fecc47d07c7245b495da0de74b854803, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/09d5e29db4a64113970c7a8435681cbe, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/ad3ff60da34c4c63b9e04e637cb98b12, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/7e3064fa3d30457ba27924d79a79cfd3, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/0e4bcbbf129c4364858a4c3e60f8017a, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/50d2eb3851ef4620805580d8544d1942, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/ee7f2534644b4569ba42766862386f47_SeqId_44_] to archive 2023-02-13 00:56:17,437 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:17,439 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/11c5a90132c14a69a638e9bfb41ce886 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/11c5a90132c14a69a638e9bfb41ce886 2023-02-13 00:56:17,440 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/584fad3180d34cee8d280a06d14d0ff4 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/584fad3180d34cee8d280a06d14d0ff4 2023-02-13 00:56:17,441 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/54af0e01ac494892ae515295e2f7b654 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/54af0e01ac494892ae515295e2f7b654 2023-02-13 00:56:17,441 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/fecc47d07c7245b495da0de74b854803 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/fecc47d07c7245b495da0de74b854803 2023-02-13 00:56:17,442 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/09d5e29db4a64113970c7a8435681cbe to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/09d5e29db4a64113970c7a8435681cbe 2023-02-13 00:56:17,443 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/ad3ff60da34c4c63b9e04e637cb98b12 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/ad3ff60da34c4c63b9e04e637cb98b12 2023-02-13 00:56:17,444 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/7e3064fa3d30457ba27924d79a79cfd3 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/7e3064fa3d30457ba27924d79a79cfd3 2023-02-13 00:56:17,445 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/0e4bcbbf129c4364858a4c3e60f8017a to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/0e4bcbbf129c4364858a4c3e60f8017a 2023-02-13 00:56:17,446 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/50d2eb3851ef4620805580d8544d1942 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/50d2eb3851ef4620805580d8544d1942 2023-02-13 00:56:17,447 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/ee7f2534644b4569ba42766862386f47_SeqId_44_ to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/ee7f2534644b4569ba42766862386f47_SeqId_44_ 2023-02-13 00:56:17,801 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231440ac144a9874ed296ec650a6c9aab0c to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231440ac144a9874ed296ec650a6c9aab0c 2023-02-13 00:56:17,802 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 00:56:17,823 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/1ca2fe651d7d43a5b81477a30eb444be first=Optional[r1] last=Optional[r4] 2023-02-13 00:56:17,824 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/1ca2fe651d7d43a5b81477a30eb444be}] 2023-02-13 00:56:17,827 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/1ca2fe651d7d43a5b81477a30eb444be for inclusion in 742bdf451b644123e692d39c2d4cda96/family1 2023-02-13 00:56:17,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-02-13 00:56:17,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 00:56:17,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for 742bdf451b644123e692d39c2d4cda96: 2023-02-13 00:56:17,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/1ca2fe651d7d43a5b81477a30eb444be to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__oalf30s7n3qc8gcn8fkmplhtqd79l9am6vt1qh3fdhi50807uo39o3d0c3lh4096/family1/1ca2fe651d7d43a5b81477a30eb444be 2023-02-13 00:56:17,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__oalf30s7n3qc8gcn8fkmplhtqd79l9am6vt1qh3fdhi50807uo39o3d0c3lh4096/family1/1ca2fe651d7d43a5b81477a30eb444be as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/7142303d63a24912913923c94cc19b28_SeqId_47_ 2023-02-13 00:56:17,833 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__oalf30s7n3qc8gcn8fkmplhtqd79l9am6vt1qh3fdhi50807uo39o3d0c3lh4096/family1/1ca2fe651d7d43a5b81477a30eb444be into 742bdf451b644123e692d39c2d4cda96/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/7142303d63a24912913923c94cc19b28_SeqId_47_ - updating store file list. 2023-02-13 00:56:17,835 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/7142303d63a24912913923c94cc19b28_SeqId_47_ into 742bdf451b644123e692d39c2d4cda96/family1 2023-02-13 00:56:17,835 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__oalf30s7n3qc8gcn8fkmplhtqd79l9am6vt1qh3fdhi50807uo39o3d0c3lh4096/family1/1ca2fe651d7d43a5b81477a30eb444be into 742bdf451b644123e692d39c2d4cda96/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/7142303d63a24912913923c94cc19b28_SeqId_47_) 2023-02-13 00:56:17,835 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__oalf30s7n3qc8gcn8fkmplhtqd79l9am6vt1qh3fdhi50807uo39o3d0c3lh4096/family1/1ca2fe651d7d43a5b81477a30eb444be 2023-02-13 00:56:17,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-13 00:56:17,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-13 00:56:17,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(1912): 742bdf451b644123e692d39c2d4cda96/family1 is initiating minor compaction (all files) 2023-02-13 00:56:17,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3b571465; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:56:17,836 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 742bdf451b644123e692d39c2d4cda96/family1 in testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:17,836 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/794c0a2a6d4143888ad413bbbb7f9d55, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/de3f2775949b497fb0198ea52c95a901, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/7142303d63a24912913923c94cc19b28_SeqId_47_] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/.tmp, totalSize=178.1 K 2023-02-13 00:56:17,837 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 794c0a2a6d4143888ad413bbbb7f9d55, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878845884 2023-02-13 00:56:17,837 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting de3f2775949b497fb0198ea52c95a901, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621245884 2023-02-13 00:56:17,837 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] compactions.Compactor(207): Compacting 7142303d63a24912913923c94cc19b28_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965245884 2023-02-13 00:56:17,838 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 00:56:17,838 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:17,839 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512016d3fd612122f4de1846982a9b0572759 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512016d3fd612122f4de1846982a9b0572759 2023-02-13 00:56:17,840 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120509a4853cad694b519af9a21802c54237 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120509a4853cad694b519af9a21802c54237 2023-02-13 00:56:17,841 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015122808bfddc357ae4c3391ac8d288b955539 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015122808bfddc357ae4c3391ac8d288b955539 2023-02-13 00:56:17,842 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512318bf236815d9e4a4cae7588e57287c8b8 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512318bf236815d9e4a4cae7588e57287c8b8 2023-02-13 00:56:17,842 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-02-13 00:56:17,842 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-02-13 00:56:17,843 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 00:56:17,843 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 00:56:17,850 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] throttle.PressureAwareThroughputController(145): 742bdf451b644123e692d39c2d4cda96#family1#compaction#166 average throughput is 22.23 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:56:18,679 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113008ae20fcc9e34dbf8fd23ca2966182ff to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113008ae20fcc9e34dbf8fd23ca2966182ff 2023-02-13 00:56:18,680 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 00:56:18,702 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/7652feb748af474a90802ab95111f9d4 first=Optional[r00] last=Optional[r8] 2023-02-13 00:56:18,703 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/7652feb748af474a90802ab95111f9d4}] 2023-02-13 00:56:18,704 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/7652feb748af474a90802ab95111f9d4 for inclusion in 742bdf451b644123e692d39c2d4cda96/family1 2023-02-13 00:56:18,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-02-13 00:56:18,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 00:56:18,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for 742bdf451b644123e692d39c2d4cda96: 2023-02-13 00:56:18,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/7652feb748af474a90802ab95111f9d4 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__74vedntdmotqahe6g9rhne282h4elohbj6orc984gdrdaf23efnb46pee0id64o7/family1/7652feb748af474a90802ab95111f9d4 2023-02-13 00:56:18,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__74vedntdmotqahe6g9rhne282h4elohbj6orc984gdrdaf23efnb46pee0id64o7/family1/7652feb748af474a90802ab95111f9d4 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/c4f834a8b2ab4bd7b93fc98f37f3dd55_SeqId_49_ 2023-02-13 00:56:18,709 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__74vedntdmotqahe6g9rhne282h4elohbj6orc984gdrdaf23efnb46pee0id64o7/family1/7652feb748af474a90802ab95111f9d4 into 742bdf451b644123e692d39c2d4cda96/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/c4f834a8b2ab4bd7b93fc98f37f3dd55_SeqId_49_ - updating store file list. 2023-02-13 00:56:18,712 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/c4f834a8b2ab4bd7b93fc98f37f3dd55_SeqId_49_ into 742bdf451b644123e692d39c2d4cda96/family1 2023-02-13 00:56:18,712 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__74vedntdmotqahe6g9rhne282h4elohbj6orc984gdrdaf23efnb46pee0id64o7/family1/7652feb748af474a90802ab95111f9d4 into 742bdf451b644123e692d39c2d4cda96/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/c4f834a8b2ab4bd7b93fc98f37f3dd55_SeqId_49_) 2023-02-13 00:56:18,712 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__74vedntdmotqahe6g9rhne282h4elohbj6orc984gdrdaf23efnb46pee0id64o7/family1/7652feb748af474a90802ab95111f9d4 2023-02-13 00:56:18,714 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 00:56:18,714 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:18,715 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511202eb114f8812445f396a1907ed8fc0e2d to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511202eb114f8812445f396a1907ed8fc0e2d 2023-02-13 00:56:18,716 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112821995ba567f7497fb6cc7e88dec5517d to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112821995ba567f7497fb6cc7e88dec5517d 2023-02-13 00:56:18,717 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113049321e6ad5724669a31a7a20ac4f57ed to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113049321e6ad5724669a31a7a20ac4f57ed 2023-02-13 00:56:18,717 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-02-13 00:56:18,717 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-13 00:56:18,718 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 00:56:18,718 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 00:56:18,718 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 00:56:18,718 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 00:56:18,718 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:56:18,720 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 00:56:18,721 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 2.2590 sec 2023-02-13 00:56:18,890 INFO [Listener at localhost/37869] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=255 (was 248) - Thread LEAK? -, OpenFileDescriptor=646 (was 624) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=104 (was 122), ProcessCount=167 (was 167), AvailableMemoryMB=3230 (was 3303) 2023-02-13 00:56:18,903 INFO [Listener at localhost/37869] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=255, OpenFileDescriptor=646, MaxFileDescriptor=60000, SystemLoadAverage=104, ProcessCount=167, AvailableMemoryMB=3229 2023-02-13 00:56:18,927 DEBUG [Listener at localhost/37869] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-02-13 00:56:18,989 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCF\x09\xB1\xE8\xC4\xE9\xF7X\xE6\x86\xB8\x93l\xF1#\x0D"\x10\xBF\x13\xAF\x9B\xDF\xD6h\x05SB\x8C=\xD5\xC9\xEB\xD0*\x10\xDB\xF1T\x83\x88\x1B\x11\xBB\xCC\xCBd\xAEFE\xD6\xF92\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-13 00:56:18,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure2.ProcedureExecutor(1029): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-02-13 00:56:18,991 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 00:56:18,991 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-02-13 00:56:18,992 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 00:56:18,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-13 00:56:18,994 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436 2023-02-13 00:56:18,994 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd 2023-02-13 00:56:18,994 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf 2023-02-13 00:56:18,995 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436 empty. 2023-02-13 00:56:18,995 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf empty. 2023-02-13 00:56:18,995 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd empty. 2023-02-13 00:56:18,995 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436 2023-02-13 00:56:18,995 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf 2023-02-13 00:56:18,995 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd 2023-02-13 00:56:18,995 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-02-13 00:56:19,003 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-02-13 00:56:19,004 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7671): creating {ENCODED => 3ec3b80d46fd5012bdeaf37435e2f8cf, NAME => 'testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCF\x09\xB1\xE8\xC4\xE9\xF7X\xE6\x86\xB8\x93l\xF1#\x0D"\x10\xBF\x13\xAF\x9B\xDF\xD6h\x05SB\x8C=\xD5\xC9\xEB\xD0*\x10\xDB\xF1T\x83\x88\x1B\x11\xBB\xCC\xCBd\xAEFE\xD6\xF92\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp 2023-02-13 00:56:19,004 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7671): creating {ENCODED => a26d5206d5dc1040fa8893199ddd4fdd, NAME => 'testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCF\x09\xB1\xE8\xC4\xE9\xF7X\xE6\x86\xB8\x93l\xF1#\x0D"\x10\xBF\x13\xAF\x9B\xDF\xD6h\x05SB\x8C=\xD5\xC9\xEB\xD0*\x10\xDB\xF1T\x83\x88\x1B\x11\xBB\xCC\xCBd\xAEFE\xD6\xF92\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp 2023-02-13 00:56:19,004 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7671): creating {ENCODED => 56fde7011992c3b17bdb846795902436, NAME => 'testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCF\x09\xB1\xE8\xC4\xE9\xF7X\xE6\x86\xB8\x93l\xF1#\x0D"\x10\xBF\x13\xAF\x9B\xDF\xD6h\x05SB\x8C=\xD5\xC9\xEB\xD0*\x10\xDB\xF1T\x83\x88\x1B\x11\xBB\xCC\xCBd\xAEFE\xD6\xF92\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/.tmp 2023-02-13 00:56:19,072 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/.tmp/family1/7b8fd5bc87d542ed84e1b7626bf64e2b as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/7b8fd5bc87d542ed84e1b7626bf64e2b 2023-02-13 00:56:19,075 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 742bdf451b644123e692d39c2d4cda96/family1 of 742bdf451b644123e692d39c2d4cda96 into 7b8fd5bc87d542ed84e1b7626bf64e2b(size=167.4 K), total size for store is 333.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 00:56:19,076 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 742bdf451b644123e692d39c2d4cda96: 2023-02-13 00:56:19,076 INFO [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1676249777836; duration=1sec 2023-02-13 00:56:19,076 DEBUG [RS:0;jenkins-hbase4:36605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:56:19,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-13 00:56:19,136 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/794c0a2a6d4143888ad413bbbb7f9d55, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/de3f2775949b497fb0198ea52c95a901, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/7142303d63a24912913923c94cc19b28_SeqId_47_] to archive 2023-02-13 00:56:19,137 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:19,138 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/794c0a2a6d4143888ad413bbbb7f9d55 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/794c0a2a6d4143888ad413bbbb7f9d55 2023-02-13 00:56:19,139 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/de3f2775949b497fb0198ea52c95a901 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/de3f2775949b497fb0198ea52c95a901 2023-02-13 00:56:19,140 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/7142303d63a24912913923c94cc19b28_SeqId_47_ to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/family1/7142303d63a24912913923c94cc19b28_SeqId_47_ 2023-02-13 00:56:19,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-13 00:56:19,420 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:56:19,420 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1603): Closing 3ec3b80d46fd5012bdeaf37435e2f8cf, disabling compactions & flushes 2023-02-13 00:56:19,420 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. 2023-02-13 00:56:19,420 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. 2023-02-13 00:56:19,420 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. after waiting 0 ms 2023-02-13 00:56:19,420 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. 2023-02-13 00:56:19,420 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. 2023-02-13 00:56:19,420 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1557): Region close journal for 3ec3b80d46fd5012bdeaf37435e2f8cf: Waiting for close lock at 1676249779420Disabling compacts and flushes for region at 1676249779420Disabling writes for close at 1676249779420Writing region close event to WAL at 1676249779420Closed at 1676249779420 2023-02-13 00:56:19,421 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:56:19,422 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1603): Closing a26d5206d5dc1040fa8893199ddd4fdd, disabling compactions & flushes 2023-02-13 00:56:19,422 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. 2023-02-13 00:56:19,422 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. 2023-02-13 00:56:19,422 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. after waiting 0 ms 2023-02-13 00:56:19,422 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. 2023-02-13 00:56:19,422 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. 2023-02-13 00:56:19,422 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1557): Region close journal for a26d5206d5dc1040fa8893199ddd4fdd: Waiting for close lock at 1676249779422Disabling compacts and flushes for region at 1676249779422Disabling writes for close at 1676249779422Writing region close event to WAL at 1676249779422Closed at 1676249779422 2023-02-13 00:56:19,427 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:56:19,427 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1603): Closing 56fde7011992c3b17bdb846795902436, disabling compactions & flushes 2023-02-13 00:56:19,427 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. 2023-02-13 00:56:19,427 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. 2023-02-13 00:56:19,428 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. after waiting 0 ms 2023-02-13 00:56:19,428 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. 2023-02-13 00:56:19,428 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. 2023-02-13 00:56:19,428 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1557): Region close journal for 56fde7011992c3b17bdb846795902436: Waiting for close lock at 1676249779427Disabling compacts and flushes for region at 1676249779427Disabling writes for close at 1676249779428 (+1 ms)Writing region close event to WAL at 1676249779428Closed at 1676249779428 2023-02-13 00:56:19,430 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 00:56:19,430 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676249779430"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249779430"}]},"ts":"1676249779430"} 2023-02-13 00:56:19,430 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676249779430"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249779430"}]},"ts":"1676249779430"} 2023-02-13 00:56:19,431 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676249779430"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249779430"}]},"ts":"1676249779430"} 2023-02-13 00:56:19,432 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-13 00:56:19,433 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 00:56:19,433 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249779433"}]},"ts":"1676249779433"} 2023-02-13 00:56:19,434 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-02-13 00:56:19,437 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=56fde7011992c3b17bdb846795902436, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=a26d5206d5dc1040fa8893199ddd4fdd, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=3ec3b80d46fd5012bdeaf37435e2f8cf, ASSIGN}] 2023-02-13 00:56:19,439 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=3ec3b80d46fd5012bdeaf37435e2f8cf, ASSIGN 2023-02-13 00:56:19,439 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=a26d5206d5dc1040fa8893199ddd4fdd, ASSIGN 2023-02-13 00:56:19,439 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=56fde7011992c3b17bdb846795902436, ASSIGN 2023-02-13 00:56:19,439 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=3ec3b80d46fd5012bdeaf37435e2f8cf, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=false 2023-02-13 00:56:19,439 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=a26d5206d5dc1040fa8893199ddd4fdd, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=false 2023-02-13 00:56:19,439 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=56fde7011992c3b17bdb846795902436, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36605,1676249622318; forceNewPlan=false, retain=false 2023-02-13 00:56:19,591 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=3ec3b80d46fd5012bdeaf37435e2f8cf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:19,591 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=a26d5206d5dc1040fa8893199ddd4fdd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:19,591 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676249779591"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249779591"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249779591"}]},"ts":"1676249779591"} 2023-02-13 00:56:19,591 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=56fde7011992c3b17bdb846795902436, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:19,591 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676249779591"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249779591"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249779591"}]},"ts":"1676249779591"} 2023-02-13 00:56:19,592 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676249779591"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249779591"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249779591"}]},"ts":"1676249779591"} 2023-02-13 00:56:19,593 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE; OpenRegionProcedure 3ec3b80d46fd5012bdeaf37435e2f8cf, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:56:19,593 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=121, ppid=117, state=RUNNABLE; OpenRegionProcedure 56fde7011992c3b17bdb846795902436, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:56:19,594 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=118, state=RUNNABLE; OpenRegionProcedure a26d5206d5dc1040fa8893199ddd4fdd, server=jenkins-hbase4.apache.org,36605,1676249622318}] 2023-02-13 00:56:19,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-13 00:56:19,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. 2023-02-13 00:56:19,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3ec3b80d46fd5012bdeaf37435e2f8cf, NAME => 'testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf.', STARTKEY => '2', ENDKEY => ''} 2023-02-13 00:56:19,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 3ec3b80d46fd5012bdeaf37435e2f8cf 2023-02-13 00:56:19,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:56:19,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 3ec3b80d46fd5012bdeaf37435e2f8cf 2023-02-13 00:56:19,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 3ec3b80d46fd5012bdeaf37435e2f8cf 2023-02-13 00:56:19,771 INFO [StoreOpener-3ec3b80d46fd5012bdeaf37435e2f8cf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 3ec3b80d46fd5012bdeaf37435e2f8cf 2023-02-13 00:56:19,773 DEBUG [StoreOpener-3ec3b80d46fd5012bdeaf37435e2f8cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1 2023-02-13 00:56:19,773 DEBUG [StoreOpener-3ec3b80d46fd5012bdeaf37435e2f8cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1 2023-02-13 00:56:19,774 INFO [StoreOpener-3ec3b80d46fd5012bdeaf37435e2f8cf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3ec3b80d46fd5012bdeaf37435e2f8cf columnFamilyName family1 2023-02-13 00:56:19,775 INFO [StoreOpener-3ec3b80d46fd5012bdeaf37435e2f8cf-1] regionserver.HStore(310): Store=3ec3b80d46fd5012bdeaf37435e2f8cf/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:19,776 INFO [StoreOpener-3ec3b80d46fd5012bdeaf37435e2f8cf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 3ec3b80d46fd5012bdeaf37435e2f8cf 2023-02-13 00:56:19,777 DEBUG [StoreOpener-3ec3b80d46fd5012bdeaf37435e2f8cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2 2023-02-13 00:56:19,777 DEBUG [StoreOpener-3ec3b80d46fd5012bdeaf37435e2f8cf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2 2023-02-13 00:56:19,778 INFO [StoreOpener-3ec3b80d46fd5012bdeaf37435e2f8cf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3ec3b80d46fd5012bdeaf37435e2f8cf columnFamilyName family2 2023-02-13 00:56:19,779 INFO [StoreOpener-3ec3b80d46fd5012bdeaf37435e2f8cf-1] regionserver.HStore(310): Store=3ec3b80d46fd5012bdeaf37435e2f8cf/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:19,780 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf 2023-02-13 00:56:19,780 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf 2023-02-13 00:56:19,782 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:56:19,784 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 3ec3b80d46fd5012bdeaf37435e2f8cf 2023-02-13 00:56:19,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:56:19,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 3ec3b80d46fd5012bdeaf37435e2f8cf; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65308695, jitterRate=-0.02682460844516754}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:56:19,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 3ec3b80d46fd5012bdeaf37435e2f8cf: Running coprocessor pre-open hook at 1676249779747Writing region info on filesystem at 1676249779747Initializing all the Stores at 1676249779748 (+1 ms)Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCF\x09\xB1\xE8\xC4\xE9\xF7X\xE6\x86\xB8\x93l\xF1#\x0D"\x10\xBF\x13\xAF\x9B\xDF\xD6h\x05SB\x8C=\xD5\xC9\xEB\xD0*\x10\xDB\xF1T\x83\x88\x1B\x11\xBB\xCC\xCBd\xAEFE\xD6\xF92\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249779748Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249779748Cleaning up temporary data from old regions at 1676249779781 (+33 ms)Cleaning up detritus from prior splits at 1676249779782 (+1 ms)Running coprocessor post-open hooks at 1676249779786 (+4 ms)Region opened successfully at 1676249779786 2023-02-13 00:56:19,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf., pid=120, masterSystemTime=1676249779744 2023-02-13 00:56:19,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. 2023-02-13 00:56:19,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. 2023-02-13 00:56:19,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. 2023-02-13 00:56:19,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 56fde7011992c3b17bdb846795902436, NAME => 'testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436.', STARTKEY => '', ENDKEY => '1'} 2023-02-13 00:56:19,789 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=3ec3b80d46fd5012bdeaf37435e2f8cf, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:19,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 56fde7011992c3b17bdb846795902436 2023-02-13 00:56:19,789 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676249779788"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249779788"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249779788"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249779788"}]},"ts":"1676249779788"} 2023-02-13 00:56:19,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:56:19,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 56fde7011992c3b17bdb846795902436 2023-02-13 00:56:19,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 56fde7011992c3b17bdb846795902436 2023-02-13 00:56:19,791 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-02-13 00:56:19,791 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; OpenRegionProcedure 3ec3b80d46fd5012bdeaf37435e2f8cf, server=jenkins-hbase4.apache.org,36605,1676249622318 in 197 msec 2023-02-13 00:56:19,792 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=3ec3b80d46fd5012bdeaf37435e2f8cf, ASSIGN in 354 msec 2023-02-13 00:56:19,811 INFO [StoreOpener-56fde7011992c3b17bdb846795902436-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 56fde7011992c3b17bdb846795902436 2023-02-13 00:56:19,812 DEBUG [StoreOpener-56fde7011992c3b17bdb846795902436-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1 2023-02-13 00:56:19,813 DEBUG [StoreOpener-56fde7011992c3b17bdb846795902436-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1 2023-02-13 00:56:19,813 INFO [StoreOpener-56fde7011992c3b17bdb846795902436-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 56fde7011992c3b17bdb846795902436 columnFamilyName family1 2023-02-13 00:56:19,814 INFO [StoreOpener-56fde7011992c3b17bdb846795902436-1] regionserver.HStore(310): Store=56fde7011992c3b17bdb846795902436/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:19,814 INFO [StoreOpener-56fde7011992c3b17bdb846795902436-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 56fde7011992c3b17bdb846795902436 2023-02-13 00:56:19,815 DEBUG [StoreOpener-56fde7011992c3b17bdb846795902436-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2 2023-02-13 00:56:19,815 DEBUG [StoreOpener-56fde7011992c3b17bdb846795902436-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2 2023-02-13 00:56:19,815 INFO [StoreOpener-56fde7011992c3b17bdb846795902436-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 56fde7011992c3b17bdb846795902436 columnFamilyName family2 2023-02-13 00:56:19,816 INFO [StoreOpener-56fde7011992c3b17bdb846795902436-1] regionserver.HStore(310): Store=56fde7011992c3b17bdb846795902436/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:19,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436 2023-02-13 00:56:19,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436 2023-02-13 00:56:19,818 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:56:19,818 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 56fde7011992c3b17bdb846795902436 2023-02-13 00:56:19,820 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:56:19,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 56fde7011992c3b17bdb846795902436; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62406047, jitterRate=-0.07007743418216705}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:56:19,820 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 56fde7011992c3b17bdb846795902436: Running coprocessor pre-open hook at 1676249779789Writing region info on filesystem at 1676249779789Initializing all the Stores at 1676249779790 (+1 ms)Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCF\x09\xB1\xE8\xC4\xE9\xF7X\xE6\x86\xB8\x93l\xF1#\x0D"\x10\xBF\x13\xAF\x9B\xDF\xD6h\x05SB\x8C=\xD5\xC9\xEB\xD0*\x10\xDB\xF1T\x83\x88\x1B\x11\xBB\xCC\xCBd\xAEFE\xD6\xF92\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249779790Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249779790Cleaning up temporary data from old regions at 1676249779817 (+27 ms)Cleaning up detritus from prior splits at 1676249779817Running coprocessor post-open hooks at 1676249779820 (+3 ms)Region opened successfully at 1676249779820 2023-02-13 00:56:19,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436., pid=121, masterSystemTime=1676249779744 2023-02-13 00:56:19,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. 2023-02-13 00:56:19,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. 2023-02-13 00:56:19,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. 2023-02-13 00:56:19,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a26d5206d5dc1040fa8893199ddd4fdd, NAME => 'testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd.', STARTKEY => '1', ENDKEY => '2'} 2023-02-13 00:56:19,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin a26d5206d5dc1040fa8893199ddd4fdd 2023-02-13 00:56:19,822 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=56fde7011992c3b17bdb846795902436, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:19,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:56:19,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a26d5206d5dc1040fa8893199ddd4fdd 2023-02-13 00:56:19,822 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676249779822"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249779822"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249779822"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249779822"}]},"ts":"1676249779822"} 2023-02-13 00:56:19,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a26d5206d5dc1040fa8893199ddd4fdd 2023-02-13 00:56:19,825 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=117 2023-02-13 00:56:19,825 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=117, state=SUCCESS; OpenRegionProcedure 56fde7011992c3b17bdb846795902436, server=jenkins-hbase4.apache.org,36605,1676249622318 in 231 msec 2023-02-13 00:56:19,825 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=56fde7011992c3b17bdb846795902436, ASSIGN in 388 msec 2023-02-13 00:56:19,842 INFO [StoreOpener-a26d5206d5dc1040fa8893199ddd4fdd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a26d5206d5dc1040fa8893199ddd4fdd 2023-02-13 00:56:19,843 DEBUG [StoreOpener-a26d5206d5dc1040fa8893199ddd4fdd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1 2023-02-13 00:56:19,843 DEBUG [StoreOpener-a26d5206d5dc1040fa8893199ddd4fdd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1 2023-02-13 00:56:19,843 INFO [StoreOpener-a26d5206d5dc1040fa8893199ddd4fdd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a26d5206d5dc1040fa8893199ddd4fdd columnFamilyName family1 2023-02-13 00:56:19,844 INFO [StoreOpener-a26d5206d5dc1040fa8893199ddd4fdd-1] regionserver.HStore(310): Store=a26d5206d5dc1040fa8893199ddd4fdd/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:19,844 INFO [StoreOpener-a26d5206d5dc1040fa8893199ddd4fdd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region a26d5206d5dc1040fa8893199ddd4fdd 2023-02-13 00:56:19,845 DEBUG [StoreOpener-a26d5206d5dc1040fa8893199ddd4fdd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2 2023-02-13 00:56:19,845 DEBUG [StoreOpener-a26d5206d5dc1040fa8893199ddd4fdd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2 2023-02-13 00:56:19,845 INFO [StoreOpener-a26d5206d5dc1040fa8893199ddd4fdd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a26d5206d5dc1040fa8893199ddd4fdd columnFamilyName family2 2023-02-13 00:56:19,846 INFO [StoreOpener-a26d5206d5dc1040fa8893199ddd4fdd-1] regionserver.HStore(310): Store=a26d5206d5dc1040fa8893199ddd4fdd/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:19,846 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd 2023-02-13 00:56:19,847 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd 2023-02-13 00:56:19,847 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:56:19,848 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a26d5206d5dc1040fa8893199ddd4fdd 2023-02-13 00:56:19,849 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:56:19,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a26d5206d5dc1040fa8893199ddd4fdd; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70937624, jitterRate=0.05705296993255615}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:56:19,850 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a26d5206d5dc1040fa8893199ddd4fdd: Running coprocessor pre-open hook at 1676249779822Writing region info on filesystem at 1676249779822Initializing all the Stores at 1676249779823 (+1 ms)Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCF\x09\xB1\xE8\xC4\xE9\xF7X\xE6\x86\xB8\x93l\xF1#\x0D"\x10\xBF\x13\xAF\x9B\xDF\xD6h\x05SB\x8C=\xD5\xC9\xEB\xD0*\x10\xDB\xF1T\x83\x88\x1B\x11\xBB\xCC\xCBd\xAEFE\xD6\xF92\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249779823Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676249779823Cleaning up temporary data from old regions at 1676249779847 (+24 ms)Cleaning up detritus from prior splits at 1676249779847Running coprocessor post-open hooks at 1676249779850 (+3 ms)Region opened successfully at 1676249779850 2023-02-13 00:56:19,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd., pid=122, masterSystemTime=1676249779744 2023-02-13 00:56:19,851 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. 2023-02-13 00:56:19,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. 2023-02-13 00:56:19,852 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=a26d5206d5dc1040fa8893199ddd4fdd, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:19,852 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676249779851"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249779851"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249779851"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249779851"}]},"ts":"1676249779851"} 2023-02-13 00:56:19,854 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=118 2023-02-13 00:56:19,854 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=118, state=SUCCESS; OpenRegionProcedure a26d5206d5dc1040fa8893199ddd4fdd, server=jenkins-hbase4.apache.org,36605,1676249622318 in 259 msec 2023-02-13 00:56:19,855 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-02-13 00:56:19,855 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=a26d5206d5dc1040fa8893199ddd4fdd, ASSIGN in 417 msec 2023-02-13 00:56:19,855 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 00:56:19,855 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249779855"}]},"ts":"1676249779855"} 2023-02-13 00:56:19,856 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-02-13 00:56:19,859 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 00:56:19,860 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 870 msec 2023-02-13 00:56:20,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-13 00:56:20,096 INFO [Listener at localhost/37869] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-02-13 00:56:20,101 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:56:20,102 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:56:20,102 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:56:20,104 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:20,105 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-13 00:56:20,105 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-13 00:56:20,105 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:20,105 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-13 00:56:20,105 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-13 00:56:20,105 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:20,106 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 00:56:20,110 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:20,110 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:20,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:20,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:20,110 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:20,110 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:20,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 00:56:20,110 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:20,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:20,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 00:56:20,111 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-13 00:56:20,111 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-13 00:56:20,111 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-13 00:56:20,111 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:20,111 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-13 00:56:20,111 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. 2023-02-13 00:56:20,111 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. started... 2023-02-13 00:56:20,111 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 00:56:20,111 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:20,111 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. 2023-02-13 00:56:20,112 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a26d5206d5dc1040fa8893199ddd4fdd 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 00:56:20,111 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. 2023-02-13 00:56:20,112 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. started... 2023-02-13 00:56:20,112 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. started... 2023-02-13 00:56:20,113 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 56fde7011992c3b17bdb846795902436 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 00:56:20,113 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 3ec3b80d46fd5012bdeaf37435e2f8cf 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 00:56:20,557 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213cae43e6dde274b8f8fde71d68fa2cc77 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230213cae43e6dde274b8f8fde71d68fa2cc77 2023-02-13 00:56:20,557 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213f0e0ed0494ac43efbabca707601d7fd9 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230213f0e0ed0494ac43efbabca707601d7fd9 2023-02-13 00:56:20,557 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302132e3543769df14db8abbc623ebd54e370 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302132e3543769df14db8abbc623ebd54e370 2023-02-13 00:56:20,573 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/.tmp/family1/d78615b249fb49a684b663891c9ec267 2023-02-13 00:56:20,575 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/.tmp/family1/e9998c4558394e22ae8898d363a8395f 2023-02-13 00:56:20,619 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213a60e27ec0b3140dc9343463154b35140 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230213a60e27ec0b3140dc9343463154b35140 2023-02-13 00:56:20,973 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/.tmp/family1/4505ea87a50f49998cab15abc95c7723 2023-02-13 00:56:21,018 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021350c10272b95740d88d008348ebb1766c to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023021350c10272b95740d88d008348ebb1766c 2023-02-13 00:56:21,023 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/.tmp/family2/da39a18a27304aa18d966ef5017bdf18 2023-02-13 00:56:21,026 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/.tmp/family1/d78615b249fb49a684b663891c9ec267 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/d78615b249fb49a684b663891c9ec267 2023-02-13 00:56:21,049 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/d78615b249fb49a684b663891c9ec267, entries=4, sequenceid=5, filesize=5.5 K 2023-02-13 00:56:21,049 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/.tmp/family2/da39a18a27304aa18d966ef5017bdf18 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/da39a18a27304aa18d966ef5017bdf18 2023-02-13 00:56:21,052 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/da39a18a27304aa18d966ef5017bdf18, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 00:56:21,052 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for a26d5206d5dc1040fa8893199ddd4fdd in 940ms, sequenceid=5, compaction requested=false 2023-02-13 00:56:21,053 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-02-13 00:56:21,053 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a26d5206d5dc1040fa8893199ddd4fdd: 2023-02-13 00:56:21,053 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. 2023-02-13 00:56:21,053 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 00:56:21,407 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302138f32c64b41db4d1faf0d2f10849808e1 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202302138f32c64b41db4d1faf0d2f10849808e1 2023-02-13 00:56:21,422 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/.tmp/family2/47401f819a3146f1b17115972528c4f8 2023-02-13 00:56:21,425 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/.tmp/family1/e9998c4558394e22ae8898d363a8395f as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/e9998c4558394e22ae8898d363a8395f 2023-02-13 00:56:21,449 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/e9998c4558394e22ae8898d363a8395f, entries=4, sequenceid=5, filesize=5.5 K 2023-02-13 00:56:21,449 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/.tmp/family2/47401f819a3146f1b17115972528c4f8 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/47401f819a3146f1b17115972528c4f8 2023-02-13 00:56:21,452 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/47401f819a3146f1b17115972528c4f8, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 00:56:21,453 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 3ec3b80d46fd5012bdeaf37435e2f8cf in 1340ms, sequenceid=5, compaction requested=false 2023-02-13 00:56:21,454 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 3ec3b80d46fd5012bdeaf37435e2f8cf: 2023-02-13 00:56:21,454 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. 2023-02-13 00:56:21,454 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 00:56:21,812 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/.tmp/family2/67b13c3e5f3b4ccdafee281f0015bbae 2023-02-13 00:56:21,816 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/.tmp/family1/4505ea87a50f49998cab15abc95c7723 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/4505ea87a50f49998cab15abc95c7723 2023-02-13 00:56:21,842 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/4505ea87a50f49998cab15abc95c7723, entries=4, sequenceid=5, filesize=5.5 K 2023-02-13 00:56:21,843 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/.tmp/family2/67b13c3e5f3b4ccdafee281f0015bbae as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/67b13c3e5f3b4ccdafee281f0015bbae 2023-02-13 00:56:21,847 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/67b13c3e5f3b4ccdafee281f0015bbae, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 00:56:21,848 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 56fde7011992c3b17bdb846795902436 in 1735ms, sequenceid=5, compaction requested=false 2023-02-13 00:56:21,848 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 56fde7011992c3b17bdb846795902436: 2023-02-13 00:56:21,848 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. 2023-02-13 00:56:21,848 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 00:56:21,848 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 00:56:21,848 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:21,848 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-13 00:56:21,848 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-13 00:56:21,850 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:21,850 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:21,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:21,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:21,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:21,850 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:21,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:21,850 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:21,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:21,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:21,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:21,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:21,851 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-13 00:56:21,851 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4080993e[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:21,851 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-13 00:56:21,851 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:21,852 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:21,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:21,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:21,852 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-13 00:56:21,852 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-13 00:56:21,852 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:56:21,853 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:21,853 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:21,854 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-13 00:56:21,854 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:21,854 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:21,854 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-13 00:56:21,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:21,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:21,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:21,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:21,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:21,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:21,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:21,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:21,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:21,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:21,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:56:21,856 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-13 00:56:21,856 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-13 00:56:21,857 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:21,857 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:21,857 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-13 00:56:21,857 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:21,858 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:21,858 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:21,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:21,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:21,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:21,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:21,858 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:21,858 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:21,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:21,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:21,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:21,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:21,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:21,860 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:21,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:21,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:21,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:21,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:21,862 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,862 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,862 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:21,862 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 00:56:21,862 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-13 00:56:21,863 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:21,862 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-13 00:56:21,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,862 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:21,863 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 00:56:21,863 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:21,863 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-13 00:56:21,863 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:21,863 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:21,863 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:21,863 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:21,963 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:21,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:21,965 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:56:21,965 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:56:21,965 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:56:21,968 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:21,970 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:21,970 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-13 00:56:21,970 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-13 00:56:21,970 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:21,970 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-13 00:56:21,970 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-13 00:56:21,970 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:21,971 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 00:56:21,972 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:21,972 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 00:56:21,973 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:21,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:21,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 00:56:21,973 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-13 00:56:21,973 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-13 00:56:21,973 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-13 00:56:21,973 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-13 00:56:21,974 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:21,974 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-13 00:56:21,974 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:21,974 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:21,974 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. 2023-02-13 00:56:21,974 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. started... 2023-02-13 00:56:21,974 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a26d5206d5dc1040fa8893199ddd4fdd 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 00:56:21,975 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. 2023-02-13 00:56:21,975 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 00:56:21,975 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:21,975 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. 2023-02-13 00:56:21,975 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. started... 2023-02-13 00:56:21,975 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. started... 2023-02-13 00:56:21,976 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 3ec3b80d46fd5012bdeaf37435e2f8cf 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 00:56:21,976 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 56fde7011992c3b17bdb846795902436 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 00:56:22,431 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302131881c85012b748d480e5485c36cc903f to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302131881c85012b748d480e5485c36cc903f 2023-02-13 00:56:22,432 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213ae06f20341514b099e84e58e802c4a94 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230213ae06f20341514b099e84e58e802c4a94 2023-02-13 00:56:22,432 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302139059721cec104632b836eaade5d527b1 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302139059721cec104632b836eaade5d527b1 2023-02-13 00:56:22,845 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/.tmp/family1/e19876f12f964a28ab5429808a2b938a 2023-02-13 00:56:22,846 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/.tmp/family1/f755a2fc5d9e426d9a1c2bd68945048a 2023-02-13 00:56:22,846 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/.tmp/family1/1fbdc0b10e554d39865bf5a0c9a9a9f4 2023-02-13 00:56:22,892 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302137ecb137bfb3146649e02c6b67647a87f to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202302137ecb137bfb3146649e02c6b67647a87f 2023-02-13 00:56:22,900 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/.tmp/family2/37c87208f1ef466188dc9bf1070f649a 2023-02-13 00:56:22,903 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/.tmp/family1/1fbdc0b10e554d39865bf5a0c9a9a9f4 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/1fbdc0b10e554d39865bf5a0c9a9a9f4 2023-02-13 00:56:22,925 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/1fbdc0b10e554d39865bf5a0c9a9a9f4, entries=4, sequenceid=9, filesize=5.5 K 2023-02-13 00:56:22,925 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/.tmp/family2/37c87208f1ef466188dc9bf1070f649a as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/37c87208f1ef466188dc9bf1070f649a 2023-02-13 00:56:22,928 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/37c87208f1ef466188dc9bf1070f649a, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 00:56:22,929 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 3ec3b80d46fd5012bdeaf37435e2f8cf in 954ms, sequenceid=9, compaction requested=false 2023-02-13 00:56:22,929 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 3ec3b80d46fd5012bdeaf37435e2f8cf: 2023-02-13 00:56:22,929 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. 2023-02-13 00:56:22,929 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 00:56:23,288 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021319cfe622f55f465f9f75e099c116029b to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e2023021319cfe622f55f465f9f75e099c116029b 2023-02-13 00:56:23,293 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213c840d4f2895f450ea0226b5b79c553eb to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230213c840d4f2895f450ea0226b5b79c553eb 2023-02-13 00:56:23,698 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/.tmp/family2/07e6eac3f8fb4c5ca3c54f8dfc03f6a5 2023-02-13 00:56:23,698 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/.tmp/family2/0f4e8db2f65943e59804f5e3d5a494c0 2023-02-13 00:56:23,702 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/.tmp/family1/e19876f12f964a28ab5429808a2b938a as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/e19876f12f964a28ab5429808a2b938a 2023-02-13 00:56:23,702 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/.tmp/family1/f755a2fc5d9e426d9a1c2bd68945048a as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/f755a2fc5d9e426d9a1c2bd68945048a 2023-02-13 00:56:23,724 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/e19876f12f964a28ab5429808a2b938a, entries=4, sequenceid=9, filesize=5.5 K 2023-02-13 00:56:23,725 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/f755a2fc5d9e426d9a1c2bd68945048a, entries=4, sequenceid=9, filesize=5.5 K 2023-02-13 00:56:23,725 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/.tmp/family2/0f4e8db2f65943e59804f5e3d5a494c0 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/0f4e8db2f65943e59804f5e3d5a494c0 2023-02-13 00:56:23,725 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/.tmp/family2/07e6eac3f8fb4c5ca3c54f8dfc03f6a5 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/07e6eac3f8fb4c5ca3c54f8dfc03f6a5 2023-02-13 00:56:23,729 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/0f4e8db2f65943e59804f5e3d5a494c0, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 00:56:23,729 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/07e6eac3f8fb4c5ca3c54f8dfc03f6a5, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 00:56:23,729 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for a26d5206d5dc1040fa8893199ddd4fdd in 1755ms, sequenceid=9, compaction requested=false 2023-02-13 00:56:23,729 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 56fde7011992c3b17bdb846795902436 in 1753ms, sequenceid=9, compaction requested=false 2023-02-13 00:56:23,729 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a26d5206d5dc1040fa8893199ddd4fdd: 2023-02-13 00:56:23,729 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 56fde7011992c3b17bdb846795902436: 2023-02-13 00:56:23,729 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. 2023-02-13 00:56:23,729 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. 2023-02-13 00:56:23,730 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 00:56:23,730 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 00:56:23,730 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 00:56:23,730 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:23,730 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-13 00:56:23,730 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-13 00:56:23,731 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:23,731 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:23,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:23,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:23,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:23,731 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:23,731 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:23,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:23,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:23,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:23,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:23,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:23,733 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-13 00:56:23,733 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5570f2a2[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:23,733 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-13 00:56:23,733 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:23,734 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:23,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:23,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:23,734 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-13 00:56:23,734 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-13 00:56:23,734 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:56:23,734 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:23,734 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:23,736 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-13 00:56:23,736 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:23,736 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:23,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:23,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:23,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:23,736 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-13 00:56:23,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:23,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:23,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:23,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:23,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:23,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:23,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:23,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:56:23,738 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-13 00:56:23,738 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-13 00:56:23,738 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:23,738 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:23,738 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-13 00:56:23,739 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:23,740 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:23,740 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:23,740 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:23,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:23,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:23,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:23,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:23,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:23,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:23,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:23,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:23,741 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:23,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:23,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:23,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:23,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:23,741 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:23,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:23,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:23,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:23,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:23,743 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:23,743 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:23,744 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 00:56:23,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:23,744 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-13 00:56:23,744 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-13 00:56:23,744 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 00:56:23,744 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:23,744 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:23,744 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:23,744 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:23,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:23,744 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:23,744 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:23,744 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:23,744 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-13 00:56:23,744 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:23,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:23,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:23,844 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:23,845 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:23,846 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:56:23,846 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:56:23,847 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:56:23,849 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:23,851 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:23,851 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-13 00:56:23,851 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-13 00:56:23,851 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:23,851 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-13 00:56:23,851 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-13 00:56:23,851 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:23,851 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 00:56:23,853 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:23,853 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:23,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:23,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:23,853 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:23,853 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:23,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 00:56:23,854 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:23,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:23,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 00:56:23,854 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-13 00:56:23,854 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-13 00:56:23,854 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-13 00:56:23,854 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-13 00:56:23,854 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:23,854 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-13 00:56:23,855 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. 2023-02-13 00:56:23,855 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. started... 2023-02-13 00:56:23,855 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 00:56:23,855 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:23,855 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. 2023-02-13 00:56:23,855 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. 2023-02-13 00:56:23,855 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. started... 2023-02-13 00:56:23,855 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a26d5206d5dc1040fa8893199ddd4fdd 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 00:56:23,855 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. started... 2023-02-13 00:56:23,855 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 56fde7011992c3b17bdb846795902436 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 00:56:23,855 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 3ec3b80d46fd5012bdeaf37435e2f8cf 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 00:56:23,894 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213cbd48f0929d84b609b72ce64a3a00274 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230213cbd48f0929d84b609b72ce64a3a00274 2023-02-13 00:56:23,904 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213d00bb3203ce14f6d88e2d581a103d21c to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230213d00bb3203ce14f6d88e2d581a103d21c 2023-02-13 00:56:24,301 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302139a1044dc71794e3399f8587a2c0d7ef4 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302139a1044dc71794e3399f8587a2c0d7ef4 2023-02-13 00:56:24,309 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/.tmp/family1/8bbb500c5152436092fa8a7030cb5491 2023-02-13 00:56:24,318 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/.tmp/family1/9815de436b804bb6af18f5b9d0cb3b59 2023-02-13 00:56:24,715 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/.tmp/family1/c5f85935060045bdbb35ce3ee4009d17 2023-02-13 00:56:24,744 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302132210232c2623408c96cf9601125bd750 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202302132210232c2623408c96cf9601125bd750 2023-02-13 00:56:24,761 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302133352d784a5f74039bb040752aecf40cb to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202302133352d784a5f74039bb040752aecf40cb 2023-02-13 00:56:24,773 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/.tmp/family2/b1507fb1069a4040b522d7a9ed2c75a0 2023-02-13 00:56:24,777 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/.tmp/family1/9815de436b804bb6af18f5b9d0cb3b59 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/9815de436b804bb6af18f5b9d0cb3b59 2023-02-13 00:56:24,803 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/9815de436b804bb6af18f5b9d0cb3b59, entries=4, sequenceid=13, filesize=5.5 K 2023-02-13 00:56:24,803 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/.tmp/family2/b1507fb1069a4040b522d7a9ed2c75a0 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/b1507fb1069a4040b522d7a9ed2c75a0 2023-02-13 00:56:24,806 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/b1507fb1069a4040b522d7a9ed2c75a0, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 00:56:24,807 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 56fde7011992c3b17bdb846795902436 in 952ms, sequenceid=13, compaction requested=true 2023-02-13 00:56:24,807 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 56fde7011992c3b17bdb846795902436: 2023-02-13 00:56:24,807 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. 2023-02-13 00:56:24,807 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 00:56:25,151 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/.tmp/family2/393a10fade7d458f9d8d5c03a9e27aa1 2023-02-13 00:56:25,155 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/.tmp/family1/8bbb500c5152436092fa8a7030cb5491 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/8bbb500c5152436092fa8a7030cb5491 2023-02-13 00:56:25,173 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213be77cf6a19b14b6cb81c6cd7ea178579 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230213be77cf6a19b14b6cb81c6cd7ea178579 2023-02-13 00:56:25,179 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/8bbb500c5152436092fa8a7030cb5491, entries=4, sequenceid=13, filesize=5.5 K 2023-02-13 00:56:25,179 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/.tmp/family2/393a10fade7d458f9d8d5c03a9e27aa1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/393a10fade7d458f9d8d5c03a9e27aa1 2023-02-13 00:56:25,183 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/393a10fade7d458f9d8d5c03a9e27aa1, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 00:56:25,184 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 3ec3b80d46fd5012bdeaf37435e2f8cf in 1329ms, sequenceid=13, compaction requested=true 2023-02-13 00:56:25,184 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 3ec3b80d46fd5012bdeaf37435e2f8cf: 2023-02-13 00:56:25,184 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. 2023-02-13 00:56:25,184 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 00:56:25,296 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 00:56:25,578 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/.tmp/family2/b035092002e645c79a465011567c670f 2023-02-13 00:56:25,582 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/.tmp/family1/c5f85935060045bdbb35ce3ee4009d17 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/c5f85935060045bdbb35ce3ee4009d17 2023-02-13 00:56:25,605 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/c5f85935060045bdbb35ce3ee4009d17, entries=4, sequenceid=13, filesize=5.5 K 2023-02-13 00:56:25,605 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/.tmp/family2/b035092002e645c79a465011567c670f as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/b035092002e645c79a465011567c670f 2023-02-13 00:56:25,609 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/b035092002e645c79a465011567c670f, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 00:56:25,610 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for a26d5206d5dc1040fa8893199ddd4fdd in 1755ms, sequenceid=13, compaction requested=true 2023-02-13 00:56:25,610 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a26d5206d5dc1040fa8893199ddd4fdd: 2023-02-13 00:56:25,610 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. 2023-02-13 00:56:25,610 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 00:56:25,610 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 00:56:25,610 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:25,610 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-13 00:56:25,610 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-13 00:56:25,613 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:25,613 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:25,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:25,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:25,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:25,613 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:25,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:25,613 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:25,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:25,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:25,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:25,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:25,614 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-13 00:56:25,614 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@446a4761[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:25,614 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-13 00:56:25,614 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:25,615 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:25,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:25,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:25,615 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-13 00:56:25,615 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-13 00:56:25,615 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:56:25,615 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:25,615 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:25,616 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:25,616 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-13 00:56:25,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:25,616 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:25,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:25,616 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-13 00:56:25,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:25,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:25,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:25,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:25,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:25,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:25,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:25,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:25,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:56:25,618 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-13 00:56:25,618 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-13 00:56:25,619 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:25,619 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:25,619 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-13 00:56:25,619 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:25,620 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:25,620 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:25,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:25,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:25,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:25,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:25,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:25,620 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:25,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:25,620 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:25,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:25,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:25,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:25,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:25,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:25,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:25,621 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:25,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:25,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:25,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:25,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:25,624 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:25,624 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:25,625 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 00:56:25,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:25,625 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-13 00:56:25,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:25,625 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:25,625 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:25,625 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:25,625 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 00:56:25,625 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:25,625 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:25,625 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:25,625 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:25,625 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-13 00:56:25,625 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:25,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:25,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:25,725 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:25,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:25,727 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:56:25,727 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:56:25,727 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 00:56:25,729 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:25,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:25,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-13 00:56:25,731 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-13 00:56:25,731 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:25,731 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-13 00:56:25,731 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-13 00:56:25,731 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:25,731 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 00:56:25,737 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:25,737 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:25,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:25,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:25,737 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:25,737 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:25,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 00:56:25,738 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:25,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:25,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 00:56:25,738 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-13 00:56:25,738 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-13 00:56:25,738 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-13 00:56:25,739 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-13 00:56:25,739 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:25,739 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-13 00:56:25,739 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. 2023-02-13 00:56:25,739 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. started... 2023-02-13 00:56:25,739 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. 2023-02-13 00:56:25,739 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. started... 2023-02-13 00:56:25,739 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a26d5206d5dc1040fa8893199ddd4fdd 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 00:56:25,739 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 3ec3b80d46fd5012bdeaf37435e2f8cf 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 00:56:25,748 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 00:56:25,748 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:25,748 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. 2023-02-13 00:56:25,748 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. started... 2023-02-13 00:56:25,748 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 56fde7011992c3b17bdb846795902436 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 00:56:25,797 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213cbfc695a3a184fc3b418f60606dfd459 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230213cbfc695a3a184fc3b418f60606dfd459 2023-02-13 00:56:25,803 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213676a119487b64224a26ca70cd34a0ac7 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230213676a119487b64224a26ca70cd34a0ac7 2023-02-13 00:56:26,192 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213812195daef4249d0a94ae5c588983588 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230213812195daef4249d0a94ae5c588983588 2023-02-13 00:56:26,210 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/.tmp/family1/39ec43a7c998455ea8316d5125d9c8e0 2023-02-13 00:56:26,220 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/.tmp/family1/6b1cc2c4c41f431e9913f28f23cdf5fe 2023-02-13 00:56:26,248 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021354ae2de647e44acc881a8012317e29af to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e2023021354ae2de647e44acc881a8012317e29af 2023-02-13 00:56:26,610 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/.tmp/family1/670ad93084b047c2bf7e85b4c9cbafba 2023-02-13 00:56:26,654 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/.tmp/family2/0ff4bf0c409a4b6b87752685b51a82d0 2023-02-13 00:56:26,657 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213f366298f2a724addb21db45858c7fd10 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230213f366298f2a724addb21db45858c7fd10 2023-02-13 00:56:26,660 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/.tmp/family1/39ec43a7c998455ea8316d5125d9c8e0 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/39ec43a7c998455ea8316d5125d9c8e0 2023-02-13 00:56:26,667 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/.tmp/family2/ac68059e38ab442e97aaae364a0f5340 2023-02-13 00:56:26,670 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/.tmp/family1/6b1cc2c4c41f431e9913f28f23cdf5fe as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/6b1cc2c4c41f431e9913f28f23cdf5fe 2023-02-13 00:56:26,686 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/39ec43a7c998455ea8316d5125d9c8e0, entries=4, sequenceid=17, filesize=5.5 K 2023-02-13 00:56:26,687 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/.tmp/family2/0ff4bf0c409a4b6b87752685b51a82d0 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/0ff4bf0c409a4b6b87752685b51a82d0 2023-02-13 00:56:26,694 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/0ff4bf0c409a4b6b87752685b51a82d0, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 00:56:26,695 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 56fde7011992c3b17bdb846795902436 in 947ms, sequenceid=17, compaction requested=true 2023-02-13 00:56:26,695 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 56fde7011992c3b17bdb846795902436: 2023-02-13 00:56:26,695 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. 2023-02-13 00:56:26,695 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 00:56:26,704 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/6b1cc2c4c41f431e9913f28f23cdf5fe, entries=4, sequenceid=17, filesize=5.5 K 2023-02-13 00:56:26,705 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/.tmp/family2/ac68059e38ab442e97aaae364a0f5340 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/ac68059e38ab442e97aaae364a0f5340 2023-02-13 00:56:26,709 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/ac68059e38ab442e97aaae364a0f5340, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 00:56:26,710 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for a26d5206d5dc1040fa8893199ddd4fdd in 971ms, sequenceid=17, compaction requested=true 2023-02-13 00:56:26,710 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a26d5206d5dc1040fa8893199ddd4fdd: 2023-02-13 00:56:26,710 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. 2023-02-13 00:56:26,710 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 00:56:27,043 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213605a363dc2be47ff82128108f365d979 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230213605a363dc2be47ff82128108f365d979 2023-02-13 00:56:27,448 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/.tmp/family2/2f186d19bbca408ea8a7496c5cc63727 2023-02-13 00:56:27,452 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/.tmp/family1/670ad93084b047c2bf7e85b4c9cbafba as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/670ad93084b047c2bf7e85b4c9cbafba 2023-02-13 00:56:27,483 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/670ad93084b047c2bf7e85b4c9cbafba, entries=4, sequenceid=17, filesize=5.5 K 2023-02-13 00:56:27,483 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/.tmp/family2/2f186d19bbca408ea8a7496c5cc63727 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/2f186d19bbca408ea8a7496c5cc63727 2023-02-13 00:56:27,488 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/2f186d19bbca408ea8a7496c5cc63727, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 00:56:27,488 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 3ec3b80d46fd5012bdeaf37435e2f8cf in 1749ms, sequenceid=17, compaction requested=true 2023-02-13 00:56:27,488 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 3ec3b80d46fd5012bdeaf37435e2f8cf: 2023-02-13 00:56:27,488 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. 2023-02-13 00:56:27,489 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 00:56:27,489 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 00:56:27,489 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:27,489 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-13 00:56:27,489 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-13 00:56:27,490 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:27,490 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:27,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:27,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:27,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:27,490 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:27,490 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:27,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:27,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:27,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:27,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:27,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:27,491 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-13 00:56:27,492 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2fb4c1d4[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:27,492 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-13 00:56:27,492 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:27,493 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:27,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:27,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:27,494 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-13 00:56:27,494 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-13 00:56:27,494 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:27,494 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:27,494 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:56:27,495 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-13 00:56:27,495 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:27,495 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:27,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:27,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:27,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:27,495 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-13 00:56:27,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:27,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:27,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:27,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:27,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:27,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:27,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:27,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:56:27,497 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-13 00:56:27,497 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-13 00:56:27,497 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:27,497 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:27,497 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-13 00:56:27,497 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:27,498 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:27,498 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:27,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:27,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:27,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:27,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:27,498 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:27,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:27,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:27,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:27,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:27,499 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:27,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:27,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:27,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:27,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:27,500 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:27,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:27,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:27,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:27,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:27,502 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:27,502 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:27,502 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 00:56:27,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:27,502 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-13 00:56:27,502 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:27,502 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-13 00:56:27,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:27,502 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:27,502 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 00:56:27,502 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:27,502 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:27,502 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-13 00:56:27,503 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:27,503 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:27,503 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:27,503 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:27,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:27,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:27,603 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:27,604 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:27,612 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 00:56:27,620 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 00:56:27,629 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 00:56:27,630 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 00:56:27,743 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:27,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:27,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-13 00:56:27,745 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-13 00:56:27,745 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:27,745 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-13 00:56:27,745 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-13 00:56:27,745 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:27,746 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 00:56:27,747 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:27,747 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:27,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:27,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:27,747 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:27,747 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:27,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 00:56:27,747 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:27,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:27,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 00:56:27,748 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-13 00:56:27,748 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-13 00:56:27,749 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-13 00:56:27,749 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-13 00:56:27,749 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:27,749 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-13 00:56:27,755 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. 2023-02-13 00:56:27,755 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. started... 2023-02-13 00:56:27,755 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a26d5206d5dc1040fa8893199ddd4fdd 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 00:56:27,756 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. 2023-02-13 00:56:27,756 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 00:56:27,756 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:27,756 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. started... 2023-02-13 00:56:27,756 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. 2023-02-13 00:56:27,756 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. started... 2023-02-13 00:56:27,756 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 3ec3b80d46fd5012bdeaf37435e2f8cf 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 00:56:27,756 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 56fde7011992c3b17bdb846795902436 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 00:56:27,836 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/.tmp/family1/fb95208c69db4cc291e04cdbc90c80ef 2023-02-13 00:56:27,859 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb95208c69db4cc291e04cdbc90c80ef 2023-02-13 00:56:28,234 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/.tmp/family1/d4e89819c25c4495addf3bd659660ab7 2023-02-13 00:56:28,244 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/.tmp/family1/f33855fa207c4dfea534951db698291f 2023-02-13 00:56:28,265 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d4e89819c25c4495addf3bd659660ab7 2023-02-13 00:56:28,278 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f33855fa207c4dfea534951db698291f 2023-02-13 00:56:28,295 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/.tmp/family2/8ce6ccca282a41e4a3f46994c57ec1a4 2023-02-13 00:56:28,299 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8ce6ccca282a41e4a3f46994c57ec1a4 2023-02-13 00:56:28,299 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/.tmp/family1/f33855fa207c4dfea534951db698291f as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/f33855fa207c4dfea534951db698291f 2023-02-13 00:56:28,357 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f33855fa207c4dfea534951db698291f 2023-02-13 00:56:28,357 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/f33855fa207c4dfea534951db698291f, entries=3, sequenceid=23, filesize=5.2 K 2023-02-13 00:56:28,358 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 00:56:28,358 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x20d7c20f to 127.0.0.1:54478 2023-02-13 00:56:28,358 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:56:28,358 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 00:56:28,358 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x079506e4 to 127.0.0.1:54478 2023-02-13 00:56:28,358 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:56:28,360 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/.tmp/family2/8ce6ccca282a41e4a3f46994c57ec1a4 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/8ce6ccca282a41e4a3f46994c57ec1a4 2023-02-13 00:56:28,365 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8ce6ccca282a41e4a3f46994c57ec1a4 2023-02-13 00:56:28,365 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/8ce6ccca282a41e4a3f46994c57ec1a4, entries=1, sequenceid=23, filesize=5.0 K 2023-02-13 00:56:28,366 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 56fde7011992c3b17bdb846795902436 in 610ms, sequenceid=23, compaction requested=true 2023-02-13 00:56:28,366 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 56fde7011992c3b17bdb846795902436: 2023-02-13 00:56:28,366 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. 2023-02-13 00:56:28,366 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 00:56:28,675 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/.tmp/family2/ed89944145e24571adc952cc91cdefaf 2023-02-13 00:56:28,678 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ed89944145e24571adc952cc91cdefaf 2023-02-13 00:56:28,679 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/.tmp/family1/fb95208c69db4cc291e04cdbc90c80ef as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/fb95208c69db4cc291e04cdbc90c80ef 2023-02-13 00:56:28,680 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/.tmp/family2/28d22c6bb824423c946eb9df5bf0796e 2023-02-13 00:56:28,684 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 28d22c6bb824423c946eb9df5bf0796e 2023-02-13 00:56:28,685 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/.tmp/family1/d4e89819c25c4495addf3bd659660ab7 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/d4e89819c25c4495addf3bd659660ab7 2023-02-13 00:56:28,707 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb95208c69db4cc291e04cdbc90c80ef 2023-02-13 00:56:28,707 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/fb95208c69db4cc291e04cdbc90c80ef, entries=3, sequenceid=23, filesize=5.2 K 2023-02-13 00:56:28,708 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/.tmp/family2/ed89944145e24571adc952cc91cdefaf as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/ed89944145e24571adc952cc91cdefaf 2023-02-13 00:56:28,717 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ed89944145e24571adc952cc91cdefaf 2023-02-13 00:56:28,718 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/ed89944145e24571adc952cc91cdefaf, entries=1, sequenceid=23, filesize=5.0 K 2023-02-13 00:56:28,718 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d4e89819c25c4495addf3bd659660ab7 2023-02-13 00:56:28,718 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/d4e89819c25c4495addf3bd659660ab7, entries=3, sequenceid=23, filesize=5.2 K 2023-02-13 00:56:28,718 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for a26d5206d5dc1040fa8893199ddd4fdd in 963ms, sequenceid=23, compaction requested=true 2023-02-13 00:56:28,719 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/.tmp/family2/28d22c6bb824423c946eb9df5bf0796e as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/28d22c6bb824423c946eb9df5bf0796e 2023-02-13 00:56:28,719 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a26d5206d5dc1040fa8893199ddd4fdd: 2023-02-13 00:56:28,719 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. 2023-02-13 00:56:28,719 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 00:56:28,723 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 28d22c6bb824423c946eb9df5bf0796e 2023-02-13 00:56:28,723 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/28d22c6bb824423c946eb9df5bf0796e, entries=1, sequenceid=23, filesize=5.0 K 2023-02-13 00:56:28,724 INFO [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 3ec3b80d46fd5012bdeaf37435e2f8cf in 968ms, sequenceid=23, compaction requested=true 2023-02-13 00:56:28,724 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 3ec3b80d46fd5012bdeaf37435e2f8cf: 2023-02-13 00:56:28,724 DEBUG [rs(jenkins-hbase4.apache.org,36605,1676249622318)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. 2023-02-13 00:56:28,724 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 00:56:28,724 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 00:56:28,724 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:28,724 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-13 00:56:28,724 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-13 00:56:28,726 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:28,726 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:28,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:28,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:28,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:28,726 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:28,726 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:28,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:28,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:28,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:28,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:28,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:28,728 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36605,1676249622318' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-13 00:56:28,728 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6b39f475[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:28,728 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-13 00:56:28,728 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:28,729 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:28,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:28,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:28,730 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-13 00:56:28,730 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-13 00:56:28,730 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,36605,1676249622318' in zk 2023-02-13 00:56:28,730 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:28,730 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:28,732 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:28,732 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-13 00:56:28,732 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:28,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:28,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:28,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:28,732 DEBUG [member: 'jenkins-hbase4.apache.org,36605,1676249622318' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-13 00:56:28,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:28,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:28,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:28,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:28,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:28,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:28,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:28,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,36605,1676249622318': 2023-02-13 00:56:28,735 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36605,1676249622318' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-13 00:56:28,735 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-13 00:56:28,735 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:28,735 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:28,735 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-13 00:56:28,735 INFO [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:28,737 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:28,737 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:28,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:28,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:28,737 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:28,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:28,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:28,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:28,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:28,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:28,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:28,738 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:28,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:28,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:28,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:28,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:28,739 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:28,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:28,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:28,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 00:56:28,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:28,741 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:28,741 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:28,741 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 00:56:28,741 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:28,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:28,742 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-13 00:56:28,742 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:28,742 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 00:56:28,742 DEBUG [(jenkins-hbase4.apache.org,43563,1676249621162)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:28,742 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:28,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:28,742 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:28,742 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 00:56:28,742 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 00:56:28,742 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-13 00:56:28,742 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:28,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:28,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:28,842 DEBUG [Listener at localhost/37869] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:28,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43563] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:28,843 DEBUG [Listener at localhost/37869] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 00:56:28,844 DEBUG [Listener at localhost/37869] regionserver.HStore(1912): a26d5206d5dc1040fa8893199ddd4fdd/family1 is initiating major compaction (all files) 2023-02-13 00:56:28,844 INFO [Listener at localhost/37869] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 00:56:28,844 INFO [Listener at localhost/37869] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:56:28,844 INFO [Listener at localhost/37869] regionserver.HRegion(2258): Starting compaction of a26d5206d5dc1040fa8893199ddd4fdd/family1 in testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. 2023-02-13 00:56:28,844 INFO [Listener at localhost/37869] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/d78615b249fb49a684b663891c9ec267, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/f755a2fc5d9e426d9a1c2bd68945048a, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/c5f85935060045bdbb35ce3ee4009d17, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/6b1cc2c4c41f431e9913f28f23cdf5fe, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/fb95208c69db4cc291e04cdbc90c80ef] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/.tmp, totalSize=27.1 K 2023-02-13 00:56:28,845 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting d78615b249fb49a684b663891c9ec267, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676249780102 2023-02-13 00:56:28,845 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting f755a2fc5d9e426d9a1c2bd68945048a, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676249781965 2023-02-13 00:56:28,845 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting c5f85935060045bdbb35ce3ee4009d17, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676249783846 2023-02-13 00:56:28,846 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 6b1cc2c4c41f431e9913f28f23cdf5fe, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676249785727 2023-02-13 00:56:28,846 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting fb95208c69db4cc291e04cdbc90c80ef, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 00:56:28,854 INFO [Listener at localhost/37869] throttle.PressureAwareThroughputController(145): a26d5206d5dc1040fa8893199ddd4fdd#family1#compaction#197 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:56:29,291 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230213d8b551b6fd62411eb1b573280fd61fd9_del 2023-02-13 00:56:29,291 INFO [Listener at localhost/37869] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213d8b551b6fd62411eb1b573280fd61fd9_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230213d8b551b6fd62411eb1b573280fd61fd9_del 2023-02-13 00:56:30,147 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for db69816925ff4b89ac2ab4481d0b6300 2023-02-13 00:56:30,148 DEBUG [Listener at localhost/37869] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/.tmp/family1/db69816925ff4b89ac2ab4481d0b6300 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/db69816925ff4b89ac2ab4481d0b6300 2023-02-13 00:56:30,171 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for db69816925ff4b89ac2ab4481d0b6300 2023-02-13 00:56:30,172 INFO [Listener at localhost/37869] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a26d5206d5dc1040fa8893199ddd4fdd/family1 of a26d5206d5dc1040fa8893199ddd4fdd into db69816925ff4b89ac2ab4481d0b6300(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 00:56:30,173 DEBUG [Listener at localhost/37869] regionserver.HRegion(2288): Compaction status journal for a26d5206d5dc1040fa8893199ddd4fdd: 2023-02-13 00:56:30,173 DEBUG [Listener at localhost/37869] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 00:56:30,173 DEBUG [Listener at localhost/37869] regionserver.HStore(1912): a26d5206d5dc1040fa8893199ddd4fdd/family2 is initiating major compaction (all files) 2023-02-13 00:56:30,173 INFO [Listener at localhost/37869] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 00:56:30,173 INFO [Listener at localhost/37869] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:56:30,173 INFO [Listener at localhost/37869] regionserver.HRegion(2258): Starting compaction of a26d5206d5dc1040fa8893199ddd4fdd/family2 in testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. 2023-02-13 00:56:30,173 INFO [Listener at localhost/37869] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/da39a18a27304aa18d966ef5017bdf18, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/0f4e8db2f65943e59804f5e3d5a494c0, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/b035092002e645c79a465011567c670f, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/ac68059e38ab442e97aaae364a0f5340, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/ed89944145e24571adc952cc91cdefaf] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/.tmp, totalSize=25.4 K 2023-02-13 00:56:30,174 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting da39a18a27304aa18d966ef5017bdf18, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676249780102 2023-02-13 00:56:30,174 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 0f4e8db2f65943e59804f5e3d5a494c0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676249781965 2023-02-13 00:56:30,174 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting b035092002e645c79a465011567c670f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676249783846 2023-02-13 00:56:30,175 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting ac68059e38ab442e97aaae364a0f5340, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676249785727 2023-02-13 00:56:30,175 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting ed89944145e24571adc952cc91cdefaf, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 00:56:30,183 INFO [Listener at localhost/37869] throttle.PressureAwareThroughputController(145): a26d5206d5dc1040fa8893199ddd4fdd#family2#compaction#198 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:56:30,236 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/d78615b249fb49a684b663891c9ec267, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/f755a2fc5d9e426d9a1c2bd68945048a, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/c5f85935060045bdbb35ce3ee4009d17, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/6b1cc2c4c41f431e9913f28f23cdf5fe, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/fb95208c69db4cc291e04cdbc90c80ef] to archive 2023-02-13 00:56:30,237 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:30,238 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/d78615b249fb49a684b663891c9ec267 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/d78615b249fb49a684b663891c9ec267 2023-02-13 00:56:30,239 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/f755a2fc5d9e426d9a1c2bd68945048a to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/f755a2fc5d9e426d9a1c2bd68945048a 2023-02-13 00:56:30,240 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/c5f85935060045bdbb35ce3ee4009d17 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/c5f85935060045bdbb35ce3ee4009d17 2023-02-13 00:56:30,241 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/6b1cc2c4c41f431e9913f28f23cdf5fe to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/6b1cc2c4c41f431e9913f28f23cdf5fe 2023-02-13 00:56:30,242 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/fb95208c69db4cc291e04cdbc90c80ef to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/fb95208c69db4cc291e04cdbc90c80ef 2023-02-13 00:56:30,591 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230213bd3a8a18001a4e33b4e59ff492aa5eec_del 2023-02-13 00:56:30,592 INFO [Listener at localhost/37869] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213bd3a8a18001a4e33b4e59ff492aa5eec_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230213bd3a8a18001a4e33b4e59ff492aa5eec_del 2023-02-13 00:56:31,405 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f7a3ec545b424dd8a870556e651245e2 2023-02-13 00:56:31,406 DEBUG [Listener at localhost/37869] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/.tmp/family2/f7a3ec545b424dd8a870556e651245e2 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/f7a3ec545b424dd8a870556e651245e2 2023-02-13 00:56:31,409 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f7a3ec545b424dd8a870556e651245e2 2023-02-13 00:56:31,410 INFO [Listener at localhost/37869] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a26d5206d5dc1040fa8893199ddd4fdd/family2 of a26d5206d5dc1040fa8893199ddd4fdd into f7a3ec545b424dd8a870556e651245e2(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 00:56:31,410 DEBUG [Listener at localhost/37869] regionserver.HRegion(2288): Compaction status journal for a26d5206d5dc1040fa8893199ddd4fdd: 2023-02-13 00:56:31,411 DEBUG [Listener at localhost/37869] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 00:56:31,411 DEBUG [Listener at localhost/37869] regionserver.HStore(1912): 3ec3b80d46fd5012bdeaf37435e2f8cf/family1 is initiating major compaction (all files) 2023-02-13 00:56:31,411 INFO [Listener at localhost/37869] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 00:56:31,411 INFO [Listener at localhost/37869] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:56:31,420 INFO [Listener at localhost/37869] regionserver.HRegion(2258): Starting compaction of 3ec3b80d46fd5012bdeaf37435e2f8cf/family1 in testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. 2023-02-13 00:56:31,420 INFO [Listener at localhost/37869] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/e9998c4558394e22ae8898d363a8395f, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/1fbdc0b10e554d39865bf5a0c9a9a9f4, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/8bbb500c5152436092fa8a7030cb5491, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/670ad93084b047c2bf7e85b4c9cbafba, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/d4e89819c25c4495addf3bd659660ab7] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/.tmp, totalSize=27.1 K 2023-02-13 00:56:31,420 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting e9998c4558394e22ae8898d363a8395f, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676249780102 2023-02-13 00:56:31,421 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 1fbdc0b10e554d39865bf5a0c9a9a9f4, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676249781965 2023-02-13 00:56:31,421 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 8bbb500c5152436092fa8a7030cb5491, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676249783847 2023-02-13 00:56:31,421 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 670ad93084b047c2bf7e85b4c9cbafba, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676249785727 2023-02-13 00:56:31,422 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting d4e89819c25c4495addf3bd659660ab7, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 00:56:31,430 INFO [Listener at localhost/37869] throttle.PressureAwareThroughputController(145): 3ec3b80d46fd5012bdeaf37435e2f8cf#family1#compaction#199 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:56:31,438 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/da39a18a27304aa18d966ef5017bdf18, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/0f4e8db2f65943e59804f5e3d5a494c0, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/b035092002e645c79a465011567c670f, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/ac68059e38ab442e97aaae364a0f5340, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/ed89944145e24571adc952cc91cdefaf] to archive 2023-02-13 00:56:31,439 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:31,441 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/da39a18a27304aa18d966ef5017bdf18 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/da39a18a27304aa18d966ef5017bdf18 2023-02-13 00:56:31,442 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/0f4e8db2f65943e59804f5e3d5a494c0 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/0f4e8db2f65943e59804f5e3d5a494c0 2023-02-13 00:56:31,443 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/b035092002e645c79a465011567c670f to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/b035092002e645c79a465011567c670f 2023-02-13 00:56:31,444 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/ac68059e38ab442e97aaae364a0f5340 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/ac68059e38ab442e97aaae364a0f5340 2023-02-13 00:56:31,445 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/ed89944145e24571adc952cc91cdefaf to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family2/ed89944145e24571adc952cc91cdefaf 2023-02-13 00:56:31,477 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230213ae42fd2893d840d1a1a63afe87a2a33d_del 2023-02-13 00:56:31,478 INFO [Listener at localhost/37869] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213ae42fd2893d840d1a1a63afe87a2a33d_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230213ae42fd2893d840d1a1a63afe87a2a33d_del 2023-02-13 00:56:31,545 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 20bdb346e7bd4b1888904818f0d79a60 2023-02-13 00:56:31,546 DEBUG [Listener at localhost/37869] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/.tmp/family1/20bdb346e7bd4b1888904818f0d79a60 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/20bdb346e7bd4b1888904818f0d79a60 2023-02-13 00:56:31,574 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 20bdb346e7bd4b1888904818f0d79a60 2023-02-13 00:56:31,575 INFO [Listener at localhost/37869] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 3ec3b80d46fd5012bdeaf37435e2f8cf/family1 of 3ec3b80d46fd5012bdeaf37435e2f8cf into 20bdb346e7bd4b1888904818f0d79a60(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:56:31,576 DEBUG [Listener at localhost/37869] regionserver.HRegion(2288): Compaction status journal for 3ec3b80d46fd5012bdeaf37435e2f8cf: 2023-02-13 00:56:31,576 DEBUG [Listener at localhost/37869] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 00:56:31,576 DEBUG [Listener at localhost/37869] regionserver.HStore(1912): 3ec3b80d46fd5012bdeaf37435e2f8cf/family2 is initiating major compaction (all files) 2023-02-13 00:56:31,576 INFO [Listener at localhost/37869] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 00:56:31,576 INFO [Listener at localhost/37869] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:56:31,576 INFO [Listener at localhost/37869] regionserver.HRegion(2258): Starting compaction of 3ec3b80d46fd5012bdeaf37435e2f8cf/family2 in testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. 2023-02-13 00:56:31,576 INFO [Listener at localhost/37869] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/47401f819a3146f1b17115972528c4f8, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/37c87208f1ef466188dc9bf1070f649a, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/393a10fade7d458f9d8d5c03a9e27aa1, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/2f186d19bbca408ea8a7496c5cc63727, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/28d22c6bb824423c946eb9df5bf0796e] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/.tmp, totalSize=25.4 K 2023-02-13 00:56:31,577 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 47401f819a3146f1b17115972528c4f8, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676249780102 2023-02-13 00:56:31,577 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 37c87208f1ef466188dc9bf1070f649a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676249781965 2023-02-13 00:56:31,578 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 393a10fade7d458f9d8d5c03a9e27aa1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676249783847 2023-02-13 00:56:31,578 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 2f186d19bbca408ea8a7496c5cc63727, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676249785727 2023-02-13 00:56:31,578 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 28d22c6bb824423c946eb9df5bf0796e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 00:56:31,588 INFO [Listener at localhost/37869] throttle.PressureAwareThroughputController(145): 3ec3b80d46fd5012bdeaf37435e2f8cf#family2#compaction#200 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:56:31,605 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230213b67cdf4dbb374aba83e143c86fc8424b_del 2023-02-13 00:56:31,605 INFO [Listener at localhost/37869] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213b67cdf4dbb374aba83e143c86fc8424b_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230213b67cdf4dbb374aba83e143c86fc8424b_del 2023-02-13 00:56:31,642 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/e9998c4558394e22ae8898d363a8395f, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/1fbdc0b10e554d39865bf5a0c9a9a9f4, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/8bbb500c5152436092fa8a7030cb5491, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/670ad93084b047c2bf7e85b4c9cbafba, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/d4e89819c25c4495addf3bd659660ab7] to archive 2023-02-13 00:56:31,643 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:31,645 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/e9998c4558394e22ae8898d363a8395f to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/e9998c4558394e22ae8898d363a8395f 2023-02-13 00:56:31,646 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/1fbdc0b10e554d39865bf5a0c9a9a9f4 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/1fbdc0b10e554d39865bf5a0c9a9a9f4 2023-02-13 00:56:31,647 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/8bbb500c5152436092fa8a7030cb5491 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/8bbb500c5152436092fa8a7030cb5491 2023-02-13 00:56:31,648 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/670ad93084b047c2bf7e85b4c9cbafba to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/670ad93084b047c2bf7e85b4c9cbafba 2023-02-13 00:56:31,649 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/d4e89819c25c4495addf3bd659660ab7 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/d4e89819c25c4495addf3bd659660ab7 2023-02-13 00:56:32,034 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2efb03af0d584bf2acabd1bb49e61432 2023-02-13 00:56:32,036 DEBUG [Listener at localhost/37869] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/.tmp/family2/2efb03af0d584bf2acabd1bb49e61432 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/2efb03af0d584bf2acabd1bb49e61432 2023-02-13 00:56:32,042 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2efb03af0d584bf2acabd1bb49e61432 2023-02-13 00:56:32,043 INFO [Listener at localhost/37869] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 3ec3b80d46fd5012bdeaf37435e2f8cf/family2 of 3ec3b80d46fd5012bdeaf37435e2f8cf into 2efb03af0d584bf2acabd1bb49e61432(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:56:32,043 DEBUG [Listener at localhost/37869] regionserver.HRegion(2288): Compaction status journal for 3ec3b80d46fd5012bdeaf37435e2f8cf: 2023-02-13 00:56:32,043 DEBUG [Listener at localhost/37869] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 00:56:32,043 DEBUG [Listener at localhost/37869] regionserver.HStore(1912): 56fde7011992c3b17bdb846795902436/family1 is initiating major compaction (all files) 2023-02-13 00:56:32,044 INFO [Listener at localhost/37869] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 00:56:32,044 INFO [Listener at localhost/37869] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:56:32,044 INFO [Listener at localhost/37869] regionserver.HRegion(2258): Starting compaction of 56fde7011992c3b17bdb846795902436/family1 in testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. 2023-02-13 00:56:32,044 INFO [Listener at localhost/37869] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/4505ea87a50f49998cab15abc95c7723, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/e19876f12f964a28ab5429808a2b938a, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/9815de436b804bb6af18f5b9d0cb3b59, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/39ec43a7c998455ea8316d5125d9c8e0, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/f33855fa207c4dfea534951db698291f] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/.tmp, totalSize=27.1 K 2023-02-13 00:56:32,044 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 4505ea87a50f49998cab15abc95c7723, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676249780101 2023-02-13 00:56:32,045 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/47401f819a3146f1b17115972528c4f8, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/37c87208f1ef466188dc9bf1070f649a, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/393a10fade7d458f9d8d5c03a9e27aa1, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/2f186d19bbca408ea8a7496c5cc63727, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/28d22c6bb824423c946eb9df5bf0796e] to archive 2023-02-13 00:56:32,045 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting e19876f12f964a28ab5429808a2b938a, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676249781965 2023-02-13 00:56:32,046 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:32,046 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 9815de436b804bb6af18f5b9d0cb3b59, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676249783846 2023-02-13 00:56:32,046 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 39ec43a7c998455ea8316d5125d9c8e0, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676249785727 2023-02-13 00:56:32,047 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting f33855fa207c4dfea534951db698291f, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 00:56:32,047 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/47401f819a3146f1b17115972528c4f8 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/47401f819a3146f1b17115972528c4f8 2023-02-13 00:56:32,049 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/37c87208f1ef466188dc9bf1070f649a to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/37c87208f1ef466188dc9bf1070f649a 2023-02-13 00:56:32,050 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/393a10fade7d458f9d8d5c03a9e27aa1 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/393a10fade7d458f9d8d5c03a9e27aa1 2023-02-13 00:56:32,051 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/2f186d19bbca408ea8a7496c5cc63727 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/2f186d19bbca408ea8a7496c5cc63727 2023-02-13 00:56:32,052 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/28d22c6bb824423c946eb9df5bf0796e to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family2/28d22c6bb824423c946eb9df5bf0796e 2023-02-13 00:56:32,057 INFO [Listener at localhost/37869] throttle.PressureAwareThroughputController(145): 56fde7011992c3b17bdb846795902436#family1#compaction#201 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:56:32,497 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023021379043e877d7f4c28a7ce4d5382af8d1d_del 2023-02-13 00:56:32,497 INFO [Listener at localhost/37869] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021379043e877d7f4c28a7ce4d5382af8d1d_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023021379043e877d7f4c28a7ce4d5382af8d1d_del 2023-02-13 00:56:32,950 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e695a987cfcb498caa4dd4b7ebc04a72 2023-02-13 00:56:32,950 DEBUG [Listener at localhost/37869] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/.tmp/family1/e695a987cfcb498caa4dd4b7ebc04a72 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/e695a987cfcb498caa4dd4b7ebc04a72 2023-02-13 00:56:32,981 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e695a987cfcb498caa4dd4b7ebc04a72 2023-02-13 00:56:32,982 INFO [Listener at localhost/37869] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 56fde7011992c3b17bdb846795902436/family1 of 56fde7011992c3b17bdb846795902436 into e695a987cfcb498caa4dd4b7ebc04a72(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:56:32,982 DEBUG [Listener at localhost/37869] regionserver.HRegion(2288): Compaction status journal for 56fde7011992c3b17bdb846795902436: 2023-02-13 00:56:32,982 DEBUG [Listener at localhost/37869] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 00:56:32,982 DEBUG [Listener at localhost/37869] regionserver.HStore(1912): 56fde7011992c3b17bdb846795902436/family2 is initiating major compaction (all files) 2023-02-13 00:56:32,982 INFO [Listener at localhost/37869] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 00:56:32,982 INFO [Listener at localhost/37869] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:56:32,983 INFO [Listener at localhost/37869] regionserver.HRegion(2258): Starting compaction of 56fde7011992c3b17bdb846795902436/family2 in testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. 2023-02-13 00:56:32,983 INFO [Listener at localhost/37869] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/67b13c3e5f3b4ccdafee281f0015bbae, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/07e6eac3f8fb4c5ca3c54f8dfc03f6a5, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/b1507fb1069a4040b522d7a9ed2c75a0, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/0ff4bf0c409a4b6b87752685b51a82d0, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/8ce6ccca282a41e4a3f46994c57ec1a4] into tmpdir=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/.tmp, totalSize=25.4 K 2023-02-13 00:56:32,983 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 67b13c3e5f3b4ccdafee281f0015bbae, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676249780101 2023-02-13 00:56:32,984 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 07e6eac3f8fb4c5ca3c54f8dfc03f6a5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676249781965 2023-02-13 00:56:32,984 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting b1507fb1069a4040b522d7a9ed2c75a0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676249783846 2023-02-13 00:56:32,984 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 0ff4bf0c409a4b6b87752685b51a82d0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676249785727 2023-02-13 00:56:32,985 DEBUG [Listener at localhost/37869] compactions.Compactor(207): Compacting 8ce6ccca282a41e4a3f46994c57ec1a4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 00:56:32,993 INFO [Listener at localhost/37869] throttle.PressureAwareThroughputController(145): 56fde7011992c3b17bdb846795902436#family2#compaction#202 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:56:33,036 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/4505ea87a50f49998cab15abc95c7723, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/e19876f12f964a28ab5429808a2b938a, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/9815de436b804bb6af18f5b9d0cb3b59, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/39ec43a7c998455ea8316d5125d9c8e0, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/f33855fa207c4dfea534951db698291f] to archive 2023-02-13 00:56:33,037 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:33,039 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/4505ea87a50f49998cab15abc95c7723 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/4505ea87a50f49998cab15abc95c7723 2023-02-13 00:56:33,040 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/e19876f12f964a28ab5429808a2b938a to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/e19876f12f964a28ab5429808a2b938a 2023-02-13 00:56:33,040 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/9815de436b804bb6af18f5b9d0cb3b59 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/9815de436b804bb6af18f5b9d0cb3b59 2023-02-13 00:56:33,041 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/39ec43a7c998455ea8316d5125d9c8e0 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/39ec43a7c998455ea8316d5125d9c8e0 2023-02-13 00:56:33,043 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/f33855fa207c4dfea534951db698291f to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/f33855fa207c4dfea534951db698291f 2023-02-13 00:56:33,400 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230213e3d2b8a637dc4ede93458da2e84cecbd_del 2023-02-13 00:56:33,401 INFO [Listener at localhost/37869] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213e3d2b8a637dc4ede93458da2e84cecbd_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230213e3d2b8a637dc4ede93458da2e84cecbd_del 2023-02-13 00:56:33,816 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 02ccc2c503a14398a7574061240fe6e9 2023-02-13 00:56:33,817 DEBUG [Listener at localhost/37869] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/.tmp/family2/02ccc2c503a14398a7574061240fe6e9 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/02ccc2c503a14398a7574061240fe6e9 2023-02-13 00:56:33,820 INFO [Listener at localhost/37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 02ccc2c503a14398a7574061240fe6e9 2023-02-13 00:56:33,821 INFO [Listener at localhost/37869] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 56fde7011992c3b17bdb846795902436/family2 of 56fde7011992c3b17bdb846795902436 into 02ccc2c503a14398a7574061240fe6e9(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:56:33,821 DEBUG [Listener at localhost/37869] regionserver.HRegion(2288): Compaction status journal for 56fde7011992c3b17bdb846795902436: 2023-02-13 00:56:33,825 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 00:56:33,828 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 00:56:33,832 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 00:56:33,833 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 00:56:33,837 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43563] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCF\x09\xB1\xE8\xC4\xE9\xF7X\xE6\x86\xB8\x93l\xF1#\x0D"\x10\xBF\x13\xAF\x9B\xDF\xD6h\x05SB\x8C=\xD5\xC9\xEB\xD0*\x10\xDB\xF1T\x83\x88\x1B\x11\xBB\xCC\xCBd\xAEFE\xD6\xF92\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-02-13 00:56:33,842 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/67b13c3e5f3b4ccdafee281f0015bbae, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/07e6eac3f8fb4c5ca3c54f8dfc03f6a5, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/b1507fb1069a4040b522d7a9ed2c75a0, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/0ff4bf0c409a4b6b87752685b51a82d0, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/8ce6ccca282a41e4a3f46994c57ec1a4] to archive 2023-02-13 00:56:33,843 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:33,844 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/67b13c3e5f3b4ccdafee281f0015bbae to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/67b13c3e5f3b4ccdafee281f0015bbae 2023-02-13 00:56:33,845 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/07e6eac3f8fb4c5ca3c54f8dfc03f6a5 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/07e6eac3f8fb4c5ca3c54f8dfc03f6a5 2023-02-13 00:56:33,847 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/b1507fb1069a4040b522d7a9ed2c75a0 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/b1507fb1069a4040b522d7a9ed2c75a0 2023-02-13 00:56:33,848 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/0ff4bf0c409a4b6b87752685b51a82d0 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/0ff4bf0c409a4b6b87752685b51a82d0 2023-02-13 00:56:33,849 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/8ce6ccca282a41e4a3f46994c57ec1a4 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family2/8ce6ccca282a41e4a3f46994c57ec1a4 2023-02-13 00:56:33,859 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] procedure2.ProcedureExecutor(1029): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-13 00:56:33,860 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-13 00:56:33,861 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:56:33,861 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676250393861 2023-02-13 00:56:33,861 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 00:56:34,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-02-13 00:56:34,224 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023021379043e877d7f4c28a7ce4d5382af8d1d_del 2023-02-13 00:56:34,244 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230213d8b551b6fd62411eb1b573280fd61fd9_del 2023-02-13 00:56:34,266 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230213ae42fd2893d840d1a1a63afe87a2a33d_del 2023-02-13 00:56:34,266 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-02-13 00:56:34,266 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] zookeeper.ReadOnlyZKClient(139): Connect 0x16841cca to 127.0.0.1:54478 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:56:34,270 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2d062bd4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:56:34,271 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230213 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-13 00:56:34,814 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213661e8aecb4e74c3d84f6fc521f345598 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230213661e8aecb4e74c3d84f6fc521f345598 2023-02-13 00:56:34,815 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230213 table='testMajorCompactionFromAdmin' column='family1' 2023-02-13 00:56:34,825 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:56:34,828 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45748, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:56:34,835 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 00:56:34,835 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 00:56:34,837 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60204, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 00:56:34,866 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230213/family1/f97594a2d0b94300a15e2648b3e6a661 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-02-13 00:56:34,866 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230213/family1/f97594a2d0b94300a15e2648b3e6a661}] 2023-02-13 00:56:34,873 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230213/family1/f97594a2d0b94300a15e2648b3e6a661 for inclusion in a26d5206d5dc1040fa8893199ddd4fdd/family1 2023-02-13 00:56:34,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-02-13 00:56:34,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-13 00:56:34,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for a26d5206d5dc1040fa8893199ddd4fdd: 2023-02-13 00:56:34,908 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230213/family1/f97594a2d0b94300a15e2648b3e6a661 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionFromAdmin__eqj7mdet21eue4e0evh6qj7h603ckc451esol04q40h7v41rhifh43nt931nsodf/family1/f97594a2d0b94300a15e2648b3e6a661 2023-02-13 00:56:34,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionFromAdmin__eqj7mdet21eue4e0evh6qj7h603ckc451esol04q40h7v41rhifh43nt931nsodf/family1/f97594a2d0b94300a15e2648b3e6a661 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/42997818ff42464a85e51dc4c9362646_SeqId_28_ 2023-02-13 00:56:34,910 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionFromAdmin__eqj7mdet21eue4e0evh6qj7h603ckc451esol04q40h7v41rhifh43nt931nsodf/family1/f97594a2d0b94300a15e2648b3e6a661 into a26d5206d5dc1040fa8893199ddd4fdd/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/42997818ff42464a85e51dc4c9362646_SeqId_28_ - updating store file list. 2023-02-13 00:56:34,944 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/42997818ff42464a85e51dc4c9362646_SeqId_28_ into a26d5206d5dc1040fa8893199ddd4fdd/family1 2023-02-13 00:56:34,944 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionFromAdmin__eqj7mdet21eue4e0evh6qj7h603ckc451esol04q40h7v41rhifh43nt931nsodf/family1/f97594a2d0b94300a15e2648b3e6a661 into a26d5206d5dc1040fa8893199ddd4fdd/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/family1/42997818ff42464a85e51dc4c9362646_SeqId_28_) 2023-02-13 00:56:34,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionFromAdmin__eqj7mdet21eue4e0evh6qj7h603ckc451esol04q40h7v41rhifh43nt931nsodf/family1/f97594a2d0b94300a15e2648b3e6a661 2023-02-13 00:56:34,947 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230213 table='testMajorCompactionFromAdmin' column='family1' 2023-02-13 00:56:34,947 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:34,949 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230213676a119487b64224a26ca70cd34a0ac7 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230213676a119487b64224a26ca70cd34a0ac7 2023-02-13 00:56:34,950 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302139a1044dc71794e3399f8587a2c0d7ef4 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302139a1044dc71794e3399f8587a2c0d7ef4 2023-02-13 00:56:34,951 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230213ae06f20341514b099e84e58e802c4a94 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230213ae06f20341514b099e84e58e802c4a94 2023-02-13 00:56:34,952 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230213cae43e6dde274b8f8fde71d68fa2cc77 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230213cae43e6dde274b8f8fde71d68fa2cc77 2023-02-13 00:56:34,952 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-13 00:56:34,952 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230213' 2023-02-13 00:56:34,953 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230213 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-13 00:56:34,953 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230213 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-13 00:56:35,928 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213538271a29af74057ba41b6827f9db371 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230213538271a29af74057ba41b6827f9db371 2023-02-13 00:56:35,929 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='testMajorCompactionFromAdmin' column='family1' 2023-02-13 00:56:35,975 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230213/family1/effedd0a38d44479b95db5bef17e6afa first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-02-13 00:56:35,976 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230213/family1/effedd0a38d44479b95db5bef17e6afa}] 2023-02-13 00:56:35,978 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230213/family1/effedd0a38d44479b95db5bef17e6afa for inclusion in 3ec3b80d46fd5012bdeaf37435e2f8cf/family1 2023-02-13 00:56:35,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-02-13 00:56:35,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-13 00:56:35,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for 3ec3b80d46fd5012bdeaf37435e2f8cf: 2023-02-13 00:56:35,999 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230213/family1/effedd0a38d44479b95db5bef17e6afa to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionFromAdmin__tkol7q3tqq7tvhg15nd8h7sdts1pgu7ncn8veou5u9d6ja6r387acdodmhlad7ht/family1/effedd0a38d44479b95db5bef17e6afa 2023-02-13 00:56:36,001 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionFromAdmin__tkol7q3tqq7tvhg15nd8h7sdts1pgu7ncn8veou5u9d6ja6r387acdodmhlad7ht/family1/effedd0a38d44479b95db5bef17e6afa as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/e94079a68088464885b23b6d30a19310_SeqId_28_ 2023-02-13 00:56:36,001 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionFromAdmin__tkol7q3tqq7tvhg15nd8h7sdts1pgu7ncn8veou5u9d6ja6r387acdodmhlad7ht/family1/effedd0a38d44479b95db5bef17e6afa into 3ec3b80d46fd5012bdeaf37435e2f8cf/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/e94079a68088464885b23b6d30a19310_SeqId_28_ - updating store file list. 2023-02-13 00:56:36,023 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/e94079a68088464885b23b6d30a19310_SeqId_28_ into 3ec3b80d46fd5012bdeaf37435e2f8cf/family1 2023-02-13 00:56:36,023 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionFromAdmin__tkol7q3tqq7tvhg15nd8h7sdts1pgu7ncn8veou5u9d6ja6r387acdodmhlad7ht/family1/effedd0a38d44479b95db5bef17e6afa into 3ec3b80d46fd5012bdeaf37435e2f8cf/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/family1/e94079a68088464885b23b6d30a19310_SeqId_28_) 2023-02-13 00:56:36,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionFromAdmin__tkol7q3tqq7tvhg15nd8h7sdts1pgu7ncn8veou5u9d6ja6r387acdodmhlad7ht/family1/effedd0a38d44479b95db5bef17e6afa 2023-02-13 00:56:36,025 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='testMajorCompactionFromAdmin' column='family1' 2023-02-13 00:56:36,025 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:36,026 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230213812195daef4249d0a94ae5c588983588 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230213812195daef4249d0a94ae5c588983588 2023-02-13 00:56:36,027 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302139059721cec104632b836eaade5d527b1 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302139059721cec104632b836eaade5d527b1 2023-02-13 00:56:36,028 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230213cbd48f0929d84b609b72ce64a3a00274 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230213cbd48f0929d84b609b72ce64a3a00274 2023-02-13 00:56:36,029 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230213f0e0ed0494ac43efbabca707601d7fd9 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230213f0e0ed0494ac43efbabca707601d7fd9 2023-02-13 00:56:36,029 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-13 00:56:36,029 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230213' 2023-02-13 00:56:36,029 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230213 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-13 00:56:36,029 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230213 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-13 00:56:36,067 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 00:56:37,004 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213c08f3d4a00d144fcbb54221e4651b146 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230213c08f3d4a00d144fcbb54221e4651b146 2023-02-13 00:56:37,005 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='testMajorCompactionFromAdmin' column='family1' 2023-02-13 00:56:37,057 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230213/family1/469d216f3fef4fb3bd8698174c6640d2 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-02-13 00:56:37,058 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436., hostname=jenkins-hbase4.apache.org,36605,1676249622318, seqNum=2 for row with hfile group [{family1,hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230213/family1/469d216f3fef4fb3bd8698174c6640d2}] 2023-02-13 00:56:37,060 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(763): Validating hfile at hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230213/family1/469d216f3fef4fb3bd8698174c6640d2 for inclusion in 56fde7011992c3b17bdb846795902436/family1 2023-02-13 00:56:37,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-02-13 00:56:37,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(777): Region bounds: first= last=1 2023-02-13 00:56:37,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegion(2445): Flush status journal for 56fde7011992c3b17bdb846795902436: 2023-02-13 00:56:37,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230213/family1/469d216f3fef4fb3bd8698174c6640d2 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionFromAdmin__9jmiaqgt5tb39t8pjq02f18ehmb827qqbrqdc3f1roma4t9ltktpp2t6nu734g5u/family1/469d216f3fef4fb3bd8698174c6640d2 2023-02-13 00:56:37,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionFromAdmin__9jmiaqgt5tb39t8pjq02f18ehmb827qqbrqdc3f1roma4t9ltktpp2t6nu734g5u/family1/469d216f3fef4fb3bd8698174c6640d2 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/d90ffbb546fb4048b7a1041acc180e6b_SeqId_28_ 2023-02-13 00:56:37,093 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(849): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionFromAdmin__9jmiaqgt5tb39t8pjq02f18ehmb827qqbrqdc3f1roma4t9ltktpp2t6nu734g5u/family1/469d216f3fef4fb3bd8698174c6640d2 into 56fde7011992c3b17bdb846795902436/family1 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/d90ffbb546fb4048b7a1041acc180e6b_SeqId_28_ - updating store file list. 2023-02-13 00:56:37,117 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(882): Loaded HFile hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/d90ffbb546fb4048b7a1041acc180e6b_SeqId_28_ into 56fde7011992c3b17bdb846795902436/family1 2023-02-13 00:56:37,118 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.HStore(855): Successfully loaded hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionFromAdmin__9jmiaqgt5tb39t8pjq02f18ehmb827qqbrqdc3f1roma4t9ltktpp2t6nu734g5u/family1/469d216f3fef4fb3bd8698174c6640d2 into 56fde7011992c3b17bdb846795902436/family1 (new location: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/family1/d90ffbb546fb4048b7a1041acc180e6b_SeqId_28_) 2023-02-13 00:56:37,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36605] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/staging/jenkins__testMajorCompactionFromAdmin__9jmiaqgt5tb39t8pjq02f18ehmb827qqbrqdc3f1roma4t9ltktpp2t6nu734g5u/family1/469d216f3fef4fb3bd8698174c6640d2 2023-02-13 00:56:37,120 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='testMajorCompactionFromAdmin' column='family1' 2023-02-13 00:56:37,121 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:37,122 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302131881c85012b748d480e5485c36cc903f to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302131881c85012b748d480e5485c36cc903f 2023-02-13 00:56:37,123 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302132e3543769df14db8abbc623ebd54e370 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302132e3543769df14db8abbc623ebd54e370 2023-02-13 00:56:37,124 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230213cbfc695a3a184fc3b418f60606dfd459 to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230213cbfc695a3a184fc3b418f60606dfd459 2023-02-13 00:56:37,125 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230213d00bb3203ce14f6d88e2d581a103d21c to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230213d00bb3203ce14f6d88e2d581a103d21c 2023-02-13 00:56:37,125 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-13 00:56:37,126 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230213' 2023-02-13 00:56:37,126 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230213 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-13 00:56:37,126 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-02-13 00:56:37,126 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-13 00:56:37,126 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(367): [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023021379043e877d7f4c28a7ce4d5382af8d1d_del] 2023-02-13 00:56:37,127 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:37,128 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023021379043e877d7f4c28a7ce4d5382af8d1d_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023021379043e877d7f4c28a7ce4d5382af8d1d_del 2023-02-13 00:56:37,128 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-13 00:56:37,128 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(367): [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230213d8b551b6fd62411eb1b573280fd61fd9_del] 2023-02-13 00:56:37,128 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:37,129 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230213d8b551b6fd62411eb1b573280fd61fd9_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230213d8b551b6fd62411eb1b573280fd61fd9_del 2023-02-13 00:56:37,129 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-13 00:56:37,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] compactions.PartitionedMobCompactor(367): [hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230213ae42fd2893d840d1a1a63afe87a2a33d_del] 2023-02-13 00:56:37,132 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:37,133 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230213ae42fd2893d840d1a1a63afe87a2a33d_del to hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230213ae42fd2893d840d1a1a63afe87a2a33d_del 2023-02-13 00:56:37,133 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-13 00:56:37,133 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676249625795] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 00:56:37,134 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-13 00:56:37,135 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 3.2750 sec 2023-02-13 00:56:37,156 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 00:56:37,157 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 00:56:37,160 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 00:56:37,160 DEBUG [Listener at localhost/37869] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 00:56:37,164 INFO [Listener at localhost/37869] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 00:56:37,198 INFO [Listener at localhost/37869] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=247 (was 255), OpenFileDescriptor=635 (was 646), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=173 (was 104) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 167), AvailableMemoryMB=3492 (was 3229) - AvailableMemoryMB LEAK? - 2023-02-13 00:56:37,198 DEBUG [Listener at localhost/37869] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7defec53 to 127.0.0.1:54478 2023-02-13 00:56:37,199 DEBUG [Listener at localhost/37869] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:56:37,199 INFO [Listener at localhost/37869] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-13 00:56:37,199 INFO [Listener at localhost/37869] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 00:56:37,199 DEBUG [Listener at localhost/37869] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0629018b to 127.0.0.1:54478 2023-02-13 00:56:37,200 DEBUG [Listener at localhost/37869] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:56:37,200 DEBUG [Listener at localhost/37869] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-13 00:56:37,200 DEBUG [Listener at localhost/37869] util.JVMClusterUtil(257): Found active master hash=1677359176, stopped=false 2023-02-13 00:56:37,200 INFO [Listener at localhost/37869] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,43563,1676249621162 2023-02-13 00:56:37,202 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 00:56:37,202 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 00:56:37,202 INFO [Listener at localhost/37869] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:56:37,202 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 00:56:37,202 DEBUG [Listener at localhost/37869] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x72b1293e to 127.0.0.1:54478 2023-02-13 00:56:37,202 DEBUG [Listener at localhost/37869] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:56:37,202 INFO [Listener at localhost/37869] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,36605,1676249622318' ***** 2023-02-13 00:56:37,202 INFO [Listener at localhost/37869] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 00:56:37,202 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 00:56:37,203 INFO [RS:0;jenkins-hbase4:36605] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 00:56:37,203 INFO [RS:0;jenkins-hbase4:36605] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 00:56:37,203 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 00:56:37,203 INFO [RS:0;jenkins-hbase4:36605] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 00:56:37,203 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(3304): Received CLOSE for 086024b75418423ee9d686cb0fb63c77 2023-02-13 00:56:37,204 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(3304): Received CLOSE for cfb81dcb75adebd110154b74cfdaab0a 2023-02-13 00:56:37,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 086024b75418423ee9d686cb0fb63c77, disabling compactions & flushes 2023-02-13 00:56:37,204 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(3304): Received CLOSE for ca49db847711a2d16c6b58efda0a70df 2023-02-13 00:56:37,204 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77. 2023-02-13 00:56:37,204 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 00:56:37,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77. 2023-02-13 00:56:37,204 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(3304): Received CLOSE for 031fe1d6fc30ca635326f901d80077b8 2023-02-13 00:56:37,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77. after waiting 0 ms 2023-02-13 00:56:37,204 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(3304): Received CLOSE for 893fe9ccf4b1790a755cd18beab22152 2023-02-13 00:56:37,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77. 2023-02-13 00:56:37,204 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(3304): Received CLOSE for e740718b17018a47b245263e5bd100e3 2023-02-13 00:56:37,204 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(3304): Received CLOSE for 914f10fbb08273a073f4cd678112ab73 2023-02-13 00:56:37,204 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(3304): Received CLOSE for 5bae73767db4da59da83135d4e2a56db 2023-02-13 00:56:37,204 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(3304): Received CLOSE for a26d5206d5dc1040fa8893199ddd4fdd 2023-02-13 00:56:37,204 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(3304): Received CLOSE for 7e22638e09b3821acf9ca191933ddf87 2023-02-13 00:56:37,204 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(3304): Received CLOSE for 3fc81cadda520d51125d8aafce375f40 2023-02-13 00:56:37,204 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(3304): Received CLOSE for a55e824ba00709d09709c5889f9c7a42 2023-02-13 00:56:37,204 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(3304): Received CLOSE for 63f9bed78ea4a19108ec7ecea54182e3 2023-02-13 00:56:37,204 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(3304): Received CLOSE for 742bdf451b644123e692d39c2d4cda96 2023-02-13 00:56:37,204 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(3304): Received CLOSE for 6b485aea7037b0c8ec399d8612e4ff1d 2023-02-13 00:56:37,204 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(3304): Received CLOSE for 3ec3b80d46fd5012bdeaf37435e2f8cf 2023-02-13 00:56:37,205 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(3304): Received CLOSE for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0 2023-02-13 00:56:37,205 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(3304): Received CLOSE for 4ead7be1fbeb7ff33b9e4c0dd1135de8 2023-02-13 00:56:37,205 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(3304): Received CLOSE for 89b71b72a624c785439c7640cd6fb013 2023-02-13 00:56:37,205 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(3304): Received CLOSE for 56fde7011992c3b17bdb846795902436 2023-02-13 00:56:37,205 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:37,205 DEBUG [RS:0;jenkins-hbase4:36605] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x39699047 to 127.0.0.1:54478 2023-02-13 00:56:37,205 DEBUG [RS:0;jenkins-hbase4:36605] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:56:37,205 INFO [RS:0;jenkins-hbase4:36605] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 00:56:37,205 INFO [RS:0;jenkins-hbase4:36605] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 00:56:37,205 INFO [RS:0;jenkins-hbase4:36605] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 00:56:37,205 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-13 00:56:37,206 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(1475): Waiting on 21 regions to close 2023-02-13 00:56:37,206 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 00:56:37,206 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 00:56:37,206 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 00:56:37,206 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 00:56:37,206 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 00:56:37,206 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-02-13 00:56:37,206 DEBUG [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(1505): Waiting on 031fe1d6fc30ca635326f901d80077b8, 086024b75418423ee9d686cb0fb63c77, 1588230740, 3ec3b80d46fd5012bdeaf37435e2f8cf, 3fc81cadda520d51125d8aafce375f40, 4ead7be1fbeb7ff33b9e4c0dd1135de8, 56fde7011992c3b17bdb846795902436, 5bae73767db4da59da83135d4e2a56db, 63f9bed78ea4a19108ec7ecea54182e3, 6b485aea7037b0c8ec399d8612e4ff1d, 742bdf451b644123e692d39c2d4cda96, 7e22638e09b3821acf9ca191933ddf87, 893fe9ccf4b1790a755cd18beab22152, 89b71b72a624c785439c7640cd6fb013, 914f10fbb08273a073f4cd678112ab73, 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, a26d5206d5dc1040fa8893199ddd4fdd, a55e824ba00709d09709c5889f9c7a42, ca49db847711a2d16c6b58efda0a70df, cfb81dcb75adebd110154b74cfdaab0a, e740718b17018a47b245263e5bd100e3 2023-02-13 00:56:37,208 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/086024b75418423ee9d686cb0fb63c77/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-02-13 00:56:37,209 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77. 2023-02-13 00:56:37,209 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 086024b75418423ee9d686cb0fb63c77: Waiting for close lock at 1676249797204Running coprocessor pre-close hooks at 1676249797204Disabling compacts and flushes for region at 1676249797204Disabling writes for close at 1676249797204Writing region close event to WAL at 1676249797205 (+1 ms)Running coprocessor post-close hooks at 1676249797208 (+3 ms)Closed at 1676249797209 (+1 ms) 2023-02-13 00:56:37,209 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1676249665738.086024b75418423ee9d686cb0fb63c77. 2023-02-13 00:56:37,209 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing cfb81dcb75adebd110154b74cfdaab0a, disabling compactions & flushes 2023-02-13 00:56:37,209 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:56:37,209 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:56:37,209 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. after waiting 0 ms 2023-02-13 00:56:37,209 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:56:37,212 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/cfb81dcb75adebd110154b74cfdaab0a/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-02-13 00:56:37,213 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:56:37,213 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for cfb81dcb75adebd110154b74cfdaab0a: Waiting for close lock at 1676249797209Running coprocessor pre-close hooks at 1676249797209Disabling compacts and flushes for region at 1676249797209Disabling writes for close at 1676249797209Writing region close event to WAL at 1676249797209Running coprocessor post-close hooks at 1676249797212 (+3 ms)Closed at 1676249797213 (+1 ms) 2023-02-13 00:56:37,213 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676249697949.cfb81dcb75adebd110154b74cfdaab0a. 2023-02-13 00:56:37,213 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing ca49db847711a2d16c6b58efda0a70df, disabling compactions & flushes 2023-02-13 00:56:37,213 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676249625052.ca49db847711a2d16c6b58efda0a70df. 2023-02-13 00:56:37,213 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676249625052.ca49db847711a2d16c6b58efda0a70df. 2023-02-13 00:56:37,213 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676249625052.ca49db847711a2d16c6b58efda0a70df. after waiting 0 ms 2023-02-13 00:56:37,213 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676249625052.ca49db847711a2d16c6b58efda0a70df. 2023-02-13 00:56:37,213 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing ca49db847711a2d16c6b58efda0a70df 1/1 column families, dataSize=109 B heapSize=592 B 2023-02-13 00:56:37,221 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/namespace/ca49db847711a2d16c6b58efda0a70df/.tmp/info/b5de25759cc447309b309b6b603becbe 2023-02-13 00:56:37,225 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/namespace/ca49db847711a2d16c6b58efda0a70df/.tmp/info/b5de25759cc447309b309b6b603becbe as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/namespace/ca49db847711a2d16c6b58efda0a70df/info/b5de25759cc447309b309b6b603becbe 2023-02-13 00:56:37,227 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/namespace/ca49db847711a2d16c6b58efda0a70df/info/b5de25759cc447309b309b6b603becbe, entries=3, sequenceid=7, filesize=4.8 K 2023-02-13 00:56:37,228 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for ca49db847711a2d16c6b58efda0a70df in 15ms, sequenceid=7, compaction requested=false 2023-02-13 00:56:37,230 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/namespace/ca49db847711a2d16c6b58efda0a70df/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-02-13 00:56:37,231 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676249625052.ca49db847711a2d16c6b58efda0a70df. 2023-02-13 00:56:37,231 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for ca49db847711a2d16c6b58efda0a70df: Waiting for close lock at 1676249797213Running coprocessor pre-close hooks at 1676249797213Disabling compacts and flushes for region at 1676249797213Disabling writes for close at 1676249797213Obtaining lock to block concurrent updates at 1676249797213Preparing flush snapshotting stores in ca49db847711a2d16c6b58efda0a70df at 1676249797213Finished memstore snapshotting hbase:namespace,,1676249625052.ca49db847711a2d16c6b58efda0a70df., syncing WAL and waiting on mvcc, flushsize=dataSize=109, getHeapSize=576, getOffHeapSize=0, getCellsCount=3 at 1676249797213Flushing stores of hbase:namespace,,1676249625052.ca49db847711a2d16c6b58efda0a70df. at 1676249797214 (+1 ms)Flushing ca49db847711a2d16c6b58efda0a70df/info: creating writer at 1676249797214Flushing ca49db847711a2d16c6b58efda0a70df/info: appending metadata at 1676249797215 (+1 ms)Flushing ca49db847711a2d16c6b58efda0a70df/info: closing flushed file at 1676249797215Flushing ca49db847711a2d16c6b58efda0a70df/info: reopening flushed file at 1676249797225 (+10 ms)Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for ca49db847711a2d16c6b58efda0a70df in 15ms, sequenceid=7, compaction requested=false at 1676249797228 (+3 ms)Writing region close event to WAL at 1676249797229 (+1 ms)Running coprocessor post-close hooks at 1676249797231 (+2 ms)Closed at 1676249797231 2023-02-13 00:56:37,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676249625052.ca49db847711a2d16c6b58efda0a70df. 2023-02-13 00:56:37,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 031fe1d6fc30ca635326f901d80077b8, disabling compactions & flushes 2023-02-13 00:56:37,232 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:56:37,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:56:37,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. after waiting 0 ms 2023-02-13 00:56:37,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:56:37,234 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithWeeklyPolicy/031fe1d6fc30ca635326f901d80077b8/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-02-13 00:56:37,235 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:56:37,235 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 031fe1d6fc30ca635326f901d80077b8: Waiting for close lock at 1676249797232Running coprocessor pre-close hooks at 1676249797232Disabling compacts and flushes for region at 1676249797232Disabling writes for close at 1676249797232Writing region close event to WAL at 1676249797232Running coprocessor post-close hooks at 1676249797234 (+2 ms)Closed at 1676249797235 (+1 ms) 2023-02-13 00:56:37,235 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1676249653640.031fe1d6fc30ca635326f901d80077b8. 2023-02-13 00:56:37,235 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 893fe9ccf4b1790a755cd18beab22152, disabling compactions & flushes 2023-02-13 00:56:37,235 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:56:37,235 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:56:37,235 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. after waiting 0 ms 2023-02-13 00:56:37,235 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:56:37,238 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyPolicy/893fe9ccf4b1790a755cd18beab22152/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-13 00:56:37,239 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:56:37,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 893fe9ccf4b1790a755cd18beab22152: Waiting for close lock at 1676249797235Running coprocessor pre-close hooks at 1676249797235Disabling compacts and flushes for region at 1676249797235Disabling writes for close at 1676249797235Writing region close event to WAL at 1676249797236 (+1 ms)Running coprocessor post-close hooks at 1676249797238 (+2 ms)Closed at 1676249797239 (+1 ms) 2023-02-13 00:56:37,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1676249670723.893fe9ccf4b1790a755cd18beab22152. 2023-02-13 00:56:37,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e740718b17018a47b245263e5bd100e3, disabling compactions & flushes 2023-02-13 00:56:37,239 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:56:37,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:56:37,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. after waiting 0 ms 2023-02-13 00:56:37,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:56:37,241 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-02-13 00:56:37,242 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/e740718b17018a47b245263e5bd100e3/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-02-13 00:56:37,242 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:56:37,243 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e740718b17018a47b245263e5bd100e3: Waiting for close lock at 1676249797239Running coprocessor pre-close hooks at 1676249797239Disabling compacts and flushes for region at 1676249797239Disabling writes for close at 1676249797239Writing region close event to WAL at 1676249797240 (+1 ms)Running coprocessor post-close hooks at 1676249797242 (+2 ms)Closed at 1676249797242 2023-02-13 00:56:37,243 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676249682568.e740718b17018a47b245263e5bd100e3. 2023-02-13 00:56:37,243 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 914f10fbb08273a073f4cd678112ab73, disabling compactions & flushes 2023-02-13 00:56:37,243 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. 2023-02-13 00:56:37,243 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. 2023-02-13 00:56:37,243 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. after waiting 0 ms 2023-02-13 00:56:37,243 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. 2023-02-13 00:56:37,246 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/914f10fbb08273a073f4cd678112ab73/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-13 00:56:37,247 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. 2023-02-13 00:56:37,247 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 914f10fbb08273a073f4cd678112ab73: Waiting for close lock at 1676249797243Running coprocessor pre-close hooks at 1676249797243Disabling compacts and flushes for region at 1676249797243Disabling writes for close at 1676249797243Writing region close event to WAL at 1676249797244 (+1 ms)Running coprocessor post-close hooks at 1676249797246 (+2 ms)Closed at 1676249797247 (+1 ms) 2023-02-13 00:56:37,247 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1676249746926.914f10fbb08273a073f4cd678112ab73. 2023-02-13 00:56:37,247 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 5bae73767db4da59da83135d4e2a56db, disabling compactions & flushes 2023-02-13 00:56:37,247 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. 2023-02-13 00:56:37,247 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. 2023-02-13 00:56:37,247 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. after waiting 0 ms 2023-02-13 00:56:37,247 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. 2023-02-13 00:56:37,250 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/5bae73767db4da59da83135d4e2a56db/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-13 00:56:37,250 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. 2023-02-13 00:56:37,250 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 5bae73767db4da59da83135d4e2a56db: Waiting for close lock at 1676249797247Running coprocessor pre-close hooks at 1676249797247Disabling compacts and flushes for region at 1676249797247Disabling writes for close at 1676249797247Writing region close event to WAL at 1676249797248 (+1 ms)Running coprocessor post-close hooks at 1676249797250 (+2 ms)Closed at 1676249797250 2023-02-13 00:56:37,250 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1676249746926.5bae73767db4da59da83135d4e2a56db. 2023-02-13 00:56:37,251 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a26d5206d5dc1040fa8893199ddd4fdd, disabling compactions & flushes 2023-02-13 00:56:37,251 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. 2023-02-13 00:56:37,251 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. 2023-02-13 00:56:37,251 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. after waiting 0 ms 2023-02-13 00:56:37,251 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. 2023-02-13 00:56:37,253 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/a26d5206d5dc1040fa8893199ddd4fdd/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-13 00:56:37,254 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. 2023-02-13 00:56:37,254 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a26d5206d5dc1040fa8893199ddd4fdd: Waiting for close lock at 1676249797251Running coprocessor pre-close hooks at 1676249797251Disabling compacts and flushes for region at 1676249797251Disabling writes for close at 1676249797251Writing region close event to WAL at 1676249797252 (+1 ms)Running coprocessor post-close hooks at 1676249797254 (+2 ms)Closed at 1676249797254 2023-02-13 00:56:37,254 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1676249778954.a26d5206d5dc1040fa8893199ddd4fdd. 2023-02-13 00:56:37,255 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 7e22638e09b3821acf9ca191933ddf87, disabling compactions & flushes 2023-02-13 00:56:37,255 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:56:37,255 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:56:37,255 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. after waiting 0 ms 2023-02-13 00:56:37,255 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:56:37,257 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithMonthlyPolicy/7e22638e09b3821acf9ca191933ddf87/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-02-13 00:56:37,258 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:56:37,258 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 7e22638e09b3821acf9ca191933ddf87: Waiting for close lock at 1676249797255Running coprocessor pre-close hooks at 1676249797255Disabling compacts and flushes for region at 1676249797255Disabling writes for close at 1676249797255Writing region close event to WAL at 1676249797255Running coprocessor post-close hooks at 1676249797258 (+3 ms)Closed at 1676249797258 2023-02-13 00:56:37,258 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1676249640497.7e22638e09b3821acf9ca191933ddf87. 2023-02-13 00:56:37,258 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 3fc81cadda520d51125d8aafce375f40, disabling compactions & flushes 2023-02-13 00:56:37,258 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40. 2023-02-13 00:56:37,258 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40. 2023-02-13 00:56:37,258 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40. after waiting 0 ms 2023-02-13 00:56:37,259 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40. 2023-02-13 00:56:37,261 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/3fc81cadda520d51125d8aafce375f40/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-02-13 00:56:37,262 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40. 2023-02-13 00:56:37,262 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 3fc81cadda520d51125d8aafce375f40: Waiting for close lock at 1676249797258Running coprocessor pre-close hooks at 1676249797258Disabling compacts and flushes for region at 1676249797258Disabling writes for close at 1676249797259 (+1 ms)Writing region close event to WAL at 1676249797259Running coprocessor post-close hooks at 1676249797262 (+3 ms)Closed at 1676249797262 2023-02-13 00:56:37,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1676249665738.3fc81cadda520d51125d8aafce375f40. 2023-02-13 00:56:37,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a55e824ba00709d09709c5889f9c7a42, disabling compactions & flushes 2023-02-13 00:56:37,263 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. 2023-02-13 00:56:37,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. 2023-02-13 00:56:37,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. after waiting 0 ms 2023-02-13 00:56:37,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. 2023-02-13 00:56:37,266 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/ns/testMinorCompaction/a55e824ba00709d09709c5889f9c7a42/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-13 00:56:37,266 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. 2023-02-13 00:56:37,266 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a55e824ba00709d09709c5889f9c7a42: Waiting for close lock at 1676249797263Running coprocessor pre-close hooks at 1676249797263Disabling compacts and flushes for region at 1676249797263Disabling writes for close at 1676249797263Writing region close event to WAL at 1676249797264 (+1 ms)Running coprocessor post-close hooks at 1676249797266 (+2 ms)Closed at 1676249797266 2023-02-13 00:56:37,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1676249746926.a55e824ba00709d09709c5889f9c7a42. 2023-02-13 00:56:37,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 63f9bed78ea4a19108ec7ecea54182e3, disabling compactions & flushes 2023-02-13 00:56:37,267 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:56:37,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:56:37,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. after waiting 0 ms 2023-02-13 00:56:37,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:56:37,269 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/63f9bed78ea4a19108ec7ecea54182e3/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-13 00:56:37,271 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:56:37,271 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 63f9bed78ea4a19108ec7ecea54182e3: Waiting for close lock at 1676249797267Running coprocessor pre-close hooks at 1676249797267Disabling compacts and flushes for region at 1676249797267Disabling writes for close at 1676249797267Writing region close event to WAL at 1676249797268 (+1 ms)Running coprocessor post-close hooks at 1676249797270 (+2 ms)Closed at 1676249797271 (+1 ms) 2023-02-13 00:56:37,271 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1676249718769.63f9bed78ea4a19108ec7ecea54182e3. 2023-02-13 00:56:37,271 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 742bdf451b644123e692d39c2d4cda96, disabling compactions & flushes 2023-02-13 00:56:37,271 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:37,271 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:37,271 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. after waiting 0 ms 2023-02-13 00:56:37,271 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:37,273 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMinorCompactionWithMonthlyPolicy/742bdf451b644123e692d39c2d4cda96/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-02-13 00:56:37,273 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:37,273 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 742bdf451b644123e692d39c2d4cda96: Waiting for close lock at 1676249797271Running coprocessor pre-close hooks at 1676249797271Disabling compacts and flushes for region at 1676249797271Disabling writes for close at 1676249797271Writing region close event to WAL at 1676249797271Running coprocessor post-close hooks at 1676249797273 (+2 ms)Closed at 1676249797273 2023-02-13 00:56:37,273 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1676249765964.742bdf451b644123e692d39c2d4cda96. 2023-02-13 00:56:37,273 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 6b485aea7037b0c8ec399d8612e4ff1d, disabling compactions & flushes 2023-02-13 00:56:37,274 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:56:37,274 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:56:37,274 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. after waiting 0 ms 2023-02-13 00:56:37,274 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:56:37,276 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/6b485aea7037b0c8ec399d8612e4ff1d/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-13 00:56:37,277 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:56:37,277 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 6b485aea7037b0c8ec399d8612e4ff1d: Waiting for close lock at 1676249797273Running coprocessor pre-close hooks at 1676249797273Disabling compacts and flushes for region at 1676249797273Disabling writes for close at 1676249797274 (+1 ms)Writing region close event to WAL at 1676249797275 (+1 ms)Running coprocessor post-close hooks at 1676249797277 (+2 ms)Closed at 1676249797277 2023-02-13 00:56:37,277 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1676249718769.6b485aea7037b0c8ec399d8612e4ff1d. 2023-02-13 00:56:37,277 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 3ec3b80d46fd5012bdeaf37435e2f8cf, disabling compactions & flushes 2023-02-13 00:56:37,277 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. 2023-02-13 00:56:37,277 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. 2023-02-13 00:56:37,277 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. after waiting 0 ms 2023-02-13 00:56:37,277 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. 2023-02-13 00:56:37,280 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/3ec3b80d46fd5012bdeaf37435e2f8cf/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-13 00:56:37,280 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. 2023-02-13 00:56:37,280 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 3ec3b80d46fd5012bdeaf37435e2f8cf: Waiting for close lock at 1676249797277Running coprocessor pre-close hooks at 1676249797277Disabling compacts and flushes for region at 1676249797277Disabling writes for close at 1676249797277Writing region close event to WAL at 1676249797278 (+1 ms)Running coprocessor post-close hooks at 1676249797280 (+2 ms)Closed at 1676249797280 2023-02-13 00:56:37,280 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1676249778954.3ec3b80d46fd5012bdeaf37435e2f8cf. 2023-02-13 00:56:37,280 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0, disabling compactions & flushes 2023-02-13 00:56:37,280 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:56:37,280 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:56:37,280 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. after waiting 0 ms 2023-02-13 00:56:37,280 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:56:37,283 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/9dbd9ef1f4f3ecd6dc6fca04bd7b98f0/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-13 00:56:37,283 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:56:37,283 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 9dbd9ef1f4f3ecd6dc6fca04bd7b98f0: Waiting for close lock at 1676249797280Running coprocessor pre-close hooks at 1676249797280Disabling compacts and flushes for region at 1676249797280Disabling writes for close at 1676249797280Writing region close event to WAL at 1676249797281 (+1 ms)Running coprocessor post-close hooks at 1676249797283 (+2 ms)Closed at 1676249797283 2023-02-13 00:56:37,283 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1676249625942.9dbd9ef1f4f3ecd6dc6fca04bd7b98f0. 2023-02-13 00:56:37,283 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 4ead7be1fbeb7ff33b9e4c0dd1135de8, disabling compactions & flushes 2023-02-13 00:56:37,283 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8. 2023-02-13 00:56:37,283 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8. 2023-02-13 00:56:37,283 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8. after waiting 0 ms 2023-02-13 00:56:37,283 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8. 2023-02-13 00:56:37,285 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testScannerOnBulkLoadRefHFiles/4ead7be1fbeb7ff33b9e4c0dd1135de8/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-02-13 00:56:37,286 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8. 2023-02-13 00:56:37,286 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 4ead7be1fbeb7ff33b9e4c0dd1135de8: Waiting for close lock at 1676249797283Running coprocessor pre-close hooks at 1676249797283Disabling compacts and flushes for region at 1676249797283Disabling writes for close at 1676249797283Writing region close event to WAL at 1676249797283Running coprocessor post-close hooks at 1676249797286 (+3 ms)Closed at 1676249797286 2023-02-13 00:56:37,286 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1676249665738.4ead7be1fbeb7ff33b9e4c0dd1135de8. 2023-02-13 00:56:37,286 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 89b71b72a624c785439c7640cd6fb013, disabling compactions & flushes 2023-02-13 00:56:37,286 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:56:37,286 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:56:37,286 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. after waiting 0 ms 2023-02-13 00:56:37,286 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:56:37,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testCompactionWithHFileLink/89b71b72a624c785439c7640cd6fb013/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-13 00:56:37,290 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:56:37,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 89b71b72a624c785439c7640cd6fb013: Waiting for close lock at 1676249797286Running coprocessor pre-close hooks at 1676249797286Disabling compacts and flushes for region at 1676249797286Disabling writes for close at 1676249797286Writing region close event to WAL at 1676249797288 (+2 ms)Running coprocessor post-close hooks at 1676249797290 (+2 ms)Closed at 1676249797290 2023-02-13 00:56:37,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1676249718769.89b71b72a624c785439c7640cd6fb013. 2023-02-13 00:56:37,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 56fde7011992c3b17bdb846795902436, disabling compactions & flushes 2023-02-13 00:56:37,290 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. 2023-02-13 00:56:37,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. 2023-02-13 00:56:37,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. after waiting 0 ms 2023-02-13 00:56:37,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. 2023-02-13 00:56:37,294 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 00:56:37,295 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/default/testMajorCompactionFromAdmin/56fde7011992c3b17bdb846795902436/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-13 00:56:37,295 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. 2023-02-13 00:56:37,295 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 56fde7011992c3b17bdb846795902436: Waiting for close lock at 1676249797290Running coprocessor pre-close hooks at 1676249797290Disabling compacts and flushes for region at 1676249797290Disabling writes for close at 1676249797290Writing region close event to WAL at 1676249797293 (+3 ms)Running coprocessor post-close hooks at 1676249797295 (+2 ms)Closed at 1676249797295 2023-02-13 00:56:37,295 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1676249778954.56fde7011992c3b17bdb846795902436. 2023-02-13 00:56:37,309 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 00:56:37,407 DEBUG [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 00:56:37,607 DEBUG [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 00:56:37,615 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740/.tmp/info/965eeb9bec6b48f4b74d60a2b09e471a 2023-02-13 00:56:37,620 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 965eeb9bec6b48f4b74d60a2b09e471a 2023-02-13 00:56:37,648 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-13 00:56:37,648 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-13 00:56:37,807 DEBUG [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 00:56:38,007 DEBUG [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 00:56:38,026 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740/.tmp/rep_barrier/672dd7e1b8be40309c2fc4a272a14841 2023-02-13 00:56:38,030 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 672dd7e1b8be40309c2fc4a272a14841 2023-02-13 00:56:38,208 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-13 00:56:38,208 DEBUG [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-13 00:56:38,208 DEBUG [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 00:56:38,408 DEBUG [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 00:56:38,436 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740/.tmp/table/8592962fd50b48a6b51c3284302d7af5 2023-02-13 00:56:38,440 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8592962fd50b48a6b51c3284302d7af5 2023-02-13 00:56:38,441 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740/.tmp/info/965eeb9bec6b48f4b74d60a2b09e471a as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740/info/965eeb9bec6b48f4b74d60a2b09e471a 2023-02-13 00:56:38,444 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 965eeb9bec6b48f4b74d60a2b09e471a 2023-02-13 00:56:38,444 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740/info/965eeb9bec6b48f4b74d60a2b09e471a, entries=226, sequenceid=130, filesize=37.9 K 2023-02-13 00:56:38,444 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740/.tmp/rep_barrier/672dd7e1b8be40309c2fc4a272a14841 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740/rep_barrier/672dd7e1b8be40309c2fc4a272a14841 2023-02-13 00:56:38,447 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 672dd7e1b8be40309c2fc4a272a14841 2023-02-13 00:56:38,447 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740/rep_barrier/672dd7e1b8be40309c2fc4a272a14841, entries=1, sequenceid=130, filesize=5.0 K 2023-02-13 00:56:38,448 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740/.tmp/table/8592962fd50b48a6b51c3284302d7af5 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740/table/8592962fd50b48a6b51c3284302d7af5 2023-02-13 00:56:38,450 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8592962fd50b48a6b51c3284302d7af5 2023-02-13 00:56:38,451 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740/table/8592962fd50b48a6b51c3284302d7af5, entries=27, sequenceid=130, filesize=6.6 K 2023-02-13 00:56:38,453 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 1247ms, sequenceid=130, compaction requested=false 2023-02-13 00:56:38,456 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-02-13 00:56:38,457 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 00:56:38,457 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 00:56:38,457 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676249797206Running coprocessor pre-close hooks at 1676249797206Disabling compacts and flushes for region at 1676249797206Disabling writes for close at 1676249797206Obtaining lock to block concurrent updates at 1676249797206Preparing flush snapshotting stores in 1588230740 at 1676249797206Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=51772, getHeapSize=80064, getOffHeapSize=0, getCellsCount=365 at 1676249797206Flushing stores of hbase:meta,,1.1588230740 at 1676249797206Flushing 1588230740/info: creating writer at 1676249797207 (+1 ms)Flushing 1588230740/info: appending metadata at 1676249797210 (+3 ms)Flushing 1588230740/info: closing flushed file at 1676249797210Flushing 1588230740/rep_barrier: creating writer at 1676249797620 (+410 ms)Flushing 1588230740/rep_barrier: appending metadata at 1676249797622 (+2 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1676249797622Flushing 1588230740/table: creating writer at 1676249798030 (+408 ms)Flushing 1588230740/table: appending metadata at 1676249798032 (+2 ms)Flushing 1588230740/table: closing flushed file at 1676249798032Flushing 1588230740/info: reopening flushed file at 1676249798441 (+409 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1676249798445 (+4 ms)Flushing 1588230740/table: reopening flushed file at 1676249798448 (+3 ms)Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 1247ms, sequenceid=130, compaction requested=false at 1676249798453 (+5 ms)Writing region close event to WAL at 1676249798454 (+1 ms)Running coprocessor post-close hooks at 1676249798457 (+3 ms)Closed at 1676249798457 2023-02-13 00:56:38,457 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-13 00:56:38,548 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 00:56:38,608 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,36605,1676249622318; all regions closed. 2023-02-13 00:56:38,615 DEBUG [RS:0;jenkins-hbase4:36605] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/oldWALs 2023-02-13 00:56:38,615 INFO [RS:0;jenkins-hbase4:36605] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36605%2C1676249622318.meta:.meta(num 1676249624781) 2023-02-13 00:56:38,619 DEBUG [RS:0;jenkins-hbase4:36605] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/oldWALs 2023-02-13 00:56:38,619 INFO [RS:0;jenkins-hbase4:36605] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36605%2C1676249622318:(num 1676249623801) 2023-02-13 00:56:38,619 DEBUG [RS:0;jenkins-hbase4:36605] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:56:38,619 INFO [RS:0;jenkins-hbase4:36605] regionserver.LeaseManager(133): Closed leases 2023-02-13 00:56:38,619 INFO [RS:0;jenkins-hbase4:36605] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-13 00:56:38,619 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 00:56:38,620 INFO [RS:0;jenkins-hbase4:36605] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36605 2023-02-13 00:56:38,628 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 00:56:38,628 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36605,1676249622318 2023-02-13 00:56:38,628 ERROR [Listener at localhost/37869-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@766347c3 rejected from java.util.concurrent.ThreadPoolExecutor@725ccdd7[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 00:56:38,629 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 00:56:38,629 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,36605,1676249622318] 2023-02-13 00:56:38,629 ERROR [Listener at localhost/37869-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6a12442a rejected from java.util.concurrent.ThreadPoolExecutor@725ccdd7[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 00:56:38,629 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36605,1676249622318; numProcessing=1 2023-02-13 00:56:38,631 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,36605,1676249622318 already deleted, retry=false 2023-02-13 00:56:38,631 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,36605,1676249622318 expired; onlineServers=0 2023-02-13 00:56:38,631 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,43563,1676249621162' ***** 2023-02-13 00:56:38,631 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-13 00:56:38,632 DEBUG [M:0;jenkins-hbase4:43563] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@22c77273, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 00:56:38,632 INFO [M:0;jenkins-hbase4:43563] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,43563,1676249621162 2023-02-13 00:56:38,632 INFO [M:0;jenkins-hbase4:43563] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,43563,1676249621162; all regions closed. 2023-02-13 00:56:38,632 DEBUG [M:0;jenkins-hbase4:43563] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:56:38,632 DEBUG [M:0;jenkins-hbase4:43563] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-13 00:56:38,632 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-13 00:56:38,632 DEBUG [M:0;jenkins-hbase4:43563] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-13 00:56:38,632 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676249623491] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676249623491,5,FailOnTimeoutGroup] 2023-02-13 00:56:38,632 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676249623491] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676249623491,5,FailOnTimeoutGroup] 2023-02-13 00:56:38,632 INFO [M:0;jenkins-hbase4:43563] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-13 00:56:38,632 INFO [M:0;jenkins-hbase4:43563] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-13 00:56:38,633 INFO [M:0;jenkins-hbase4:43563] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-13 00:56:38,633 DEBUG [M:0;jenkins-hbase4:43563] master.HMaster(1502): Stopping service threads 2023-02-13 00:56:38,633 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-13 00:56:38,633 INFO [M:0;jenkins-hbase4:43563] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-13 00:56:38,633 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 00:56:38,633 ERROR [M:0;jenkins-hbase4:43563] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-02-13 00:56:38,633 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 00:56:38,634 INFO [M:0;jenkins-hbase4:43563] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:56:38,634 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-13 00:56:38,634 DEBUG [M:0;jenkins-hbase4:43563] zookeeper.ZKUtil(398): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-13 00:56:38,634 WARN [M:0;jenkins-hbase4:43563] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-13 00:56:38,635 INFO [M:0;jenkins-hbase4:43563] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:56:38,635 INFO [M:0;jenkins-hbase4:43563] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-13 00:56:38,635 DEBUG [M:0;jenkins-hbase4:43563] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 00:56:38,635 INFO [M:0;jenkins-hbase4:43563] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 00:56:38,635 DEBUG [M:0;jenkins-hbase4:43563] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 00:56:38,635 DEBUG [M:0;jenkins-hbase4:43563] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 00:56:38,635 DEBUG [M:0;jenkins-hbase4:43563] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 00:56:38,635 INFO [M:0;jenkins-hbase4:43563] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.54 KB heapSize=598.29 KB 2023-02-13 00:56:38,731 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 00:56:38,731 INFO [RS:0;jenkins-hbase4:36605] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,36605,1676249622318; zookeeper connection closed. 2023-02-13 00:56:38,731 ERROR [Listener at localhost/37869-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@12850c92 rejected from java.util.concurrent.ThreadPoolExecutor@725ccdd7[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 00:56:38,731 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): regionserver:36605-0x1004b2b47c80001, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 00:56:38,731 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@14c87992] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@14c87992 2023-02-13 00:56:38,731 ERROR [Listener at localhost/37869-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2f3a8f67 rejected from java.util.concurrent.ThreadPoolExecutor@725ccdd7[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 00:56:38,731 INFO [Listener at localhost/37869] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-02-13 00:56:39,046 INFO [M:0;jenkins-hbase4:43563] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.54 KB at sequenceid=1005 (bloomFilter=true), to=hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/a11040f452874648b6c61eb27799edf3 2023-02-13 00:56:39,049 INFO [M:0;jenkins-hbase4:43563] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a11040f452874648b6c61eb27799edf3 2023-02-13 00:56:39,049 DEBUG [M:0;jenkins-hbase4:43563] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/a11040f452874648b6c61eb27799edf3 as hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/a11040f452874648b6c61eb27799edf3 2023-02-13 00:56:39,052 INFO [M:0;jenkins-hbase4:43563] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a11040f452874648b6c61eb27799edf3 2023-02-13 00:56:39,052 INFO [M:0;jenkins-hbase4:43563] regionserver.HStore(1080): Added hdfs://localhost:38857/user/jenkins/test-data/509ea967-3600-464a-32d4-73a23af51c00/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/a11040f452874648b6c61eb27799edf3, entries=123, sequenceid=1005, filesize=33.3 K 2023-02-13 00:56:39,053 INFO [M:0;jenkins-hbase4:43563] regionserver.HRegion(2947): Finished flush of dataSize ~512.54 KB/524842, heapSize ~598.27 KB/612632, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 418ms, sequenceid=1005, compaction requested=false 2023-02-13 00:56:39,053 INFO [M:0;jenkins-hbase4:43563] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 00:56:39,053 DEBUG [M:0;jenkins-hbase4:43563] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676249798635Disabling compacts and flushes for region at 1676249798635Disabling writes for close at 1676249798635Obtaining lock to block concurrent updates at 1676249798635Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676249798635Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=524842, getHeapSize=612632, getOffHeapSize=0, getCellsCount=1160 at 1676249798635Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676249798636 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676249798636Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676249798638 (+2 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676249798638Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676249799050 (+412 ms)Finished flush of dataSize ~512.54 KB/524842, heapSize ~598.27 KB/612632, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 418ms, sequenceid=1005, compaction requested=false at 1676249799053 (+3 ms)Writing region close event to WAL at 1676249799053Closed at 1676249799053 2023-02-13 00:56:39,056 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 00:56:39,056 INFO [M:0;jenkins-hbase4:43563] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-13 00:56:39,056 INFO [M:0;jenkins-hbase4:43563] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43563 2023-02-13 00:56:39,059 DEBUG [M:0;jenkins-hbase4:43563] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,43563,1676249621162 already deleted, retry=false 2023-02-13 00:56:39,161 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 00:56:39,161 INFO [M:0;jenkins-hbase4:43563] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,43563,1676249621162; zookeeper connection closed. 2023-02-13 00:56:39,161 ERROR [Listener at localhost/37869-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4166fac8 rejected from java.util.concurrent.ThreadPoolExecutor@5f3ba08[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 00:56:39,162 DEBUG [Listener at localhost/37869-EventThread] zookeeper.ZKWatcher(600): master:43563-0x1004b2b47c80000, quorum=127.0.0.1:54478, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 00:56:39,162 ERROR [Listener at localhost/37869-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@700ada3a rejected from java.util.concurrent.ThreadPoolExecutor@5f3ba08[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 00:56:39,163 WARN [Listener at localhost/37869] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 00:56:39,167 INFO [Listener at localhost/37869] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 00:56:39,274 WARN [BP-1259063294-172.31.14.131-1676249617158 heartbeating to localhost/127.0.0.1:38857] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 00:56:39,274 WARN [BP-1259063294-172.31.14.131-1676249617158 heartbeating to localhost/127.0.0.1:38857] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1259063294-172.31.14.131-1676249617158 (Datanode Uuid 2310d9d9-ed70-437e-8abe-6c0931ad74c5) service to localhost/127.0.0.1:38857 2023-02-13 00:56:39,277 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/cluster_0ce39707-0279-6124-1e28-d0d9ced157d1/dfs/data/data1/current/BP-1259063294-172.31.14.131-1676249617158] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 00:56:39,277 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/def3bcb3-ba68-52b9-5fe2-803fb9591b79/cluster_0ce39707-0279-6124-1e28-d0d9ced157d1/dfs/data/data2/current/BP-1259063294-172.31.14.131-1676249617158] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 00:56:39,309 INFO [Listener at localhost/37869] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 00:56:39,443 INFO [Listener at localhost/37869] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-13 00:56:39,535 INFO [Listener at localhost/37869] hbase.HBaseTestingUtility(1293): Minicluster is down